data.table

Fast and Memory Efficient Data Manipulation


“Sometimes I’ll start a sentence and I don’t even know where it’s going. I just hope I find it along the way.”


“I am fast. To give you a reference point, I am somewhere between a snake and a mongoose… and a panther.”

Why data.table?

  1. Concise syntax: fast to type, fast to read – “Me think, why waste time say lot word when few word do trick.”
  1. Fast speed: optimized C code under the hood – “I am faster than 80 percent of all snakes.”
  1. Memory efficient: modify by reference – “Whenever I’m about to do something, I think, “would an idiot do that?” And if they would, I do not do that thing.”

Why data.table?

  1. Careful API lifecycle management: stable, backward compatible – “I’m not superstitious, but I am a little stitious.”
  1. Community: active development and support – “Would I rather be feared or loved? Easy. Both. I want people to be afraid of how much they love me.”
  1. Feature rich: comprehensive data manipulation toolkit – “Bears, beets, Battlestar Galactica.”

Data for Examples

Loading PUMS Data

Preview to next section with Jonathan about arrow:

library(data.table)
library(dplyr)
library(arrow)

# Preview into next section: Load multiple states and years using arrow
pums <- open_dataset(here::here("data/person")) |>
  filter(year >= 2020, location %in% c("ca", "tx", "ny")) |>
  collect()
setDT(pums)

Loading PUMS Data

pums[, 1:5]
         SPORDER            RT      SERIALNO   PUMA          ST
           <int>        <fctr>        <char> <char>      <char>
      1:       1 Person Record 2021GQ0000077  03301 New York/NY
      2:       1 Person Record 2021GQ0000090  01600 New York/NY
      3:       1 Person Record 2021GQ0000091  03212 New York/NY
      4:       1 Person Record 2021GQ0000094  02100 New York/NY
      5:       1 Person Record 2021GQ0000097  04102 New York/NY
     ---                                                       
1735551:       1 Person Record 2022HU0609672  04611    Texas/TX
1735552:       2 Person Record 2022HU0609672  04611    Texas/TX
1735553:       3 Person Record 2022HU0609672  04611    Texas/TX
1735554:       1 Person Record 2022HU0609675  04302    Texas/TX
1735555:       2 Person Record 2022HU0609675  04302    Texas/TX
names(pums)
  [1] "SPORDER"   "RT"        "SERIALNO"  "PUMA"      "ST"        "ADJUST"   
  [7] "PWGTP"     "AGEP"      "CIT"       "COW"       "DDRS"      "DEYE"     
 [13] "DOUT"      "DPHY"      "DREM"      "DWRK"      "ENG"       "FER"      
 [19] "GCL"       "GCM"       "GCR"       "INTP"      "JWMNP"     "JWRIP"    
 [25] "JWTR"      "LANX"      "MAR"       "MIG"       "MIL"       "MILY"     
 [31] "MLPA"      "MLPB"      "MLPC"      "MLPD"      "MLPE"      "MLPF"     
 [37] "MLPG"      "MLPH"      "MLPI"      "MLPJ"      "MLPK"      "NWAB"     
 [43] "NWAV"      "NWLA"      "NWLK"      "NWRE"      "OIP"       "PAP"      
 [49] "REL"       "RETP"      "SCH"       "SCHG"      "SCHL"      "SEMP"     
 [55] "SEX"       "SSIP"      "SSP"       "WAGP"      "WKHP"      "WKL"      
 [61] "WKW"       "YOEP"      "UWRK"      "ANC"       "ANC1P"     "ANC2P"    
 [67] "DECADE"    "DRIVESP"   "DS"        "ESP"       "ESR"       "HISP"     
 [73] "INDP"      "JWAP"      "JWDP"      "LANP"      "MIGPUMA"   "MIGSP"    
 [79] "MSP"       "NAICSP"    "NATIVITY"  "OC"        "OCCP"      "PAOC"     
 [85] "PERNP"     "PINCP"     "POBP"      "POVPIP"    "POWPUMA"   "POWSP"    
 [91] "QTRBIR"    "RAC1P"     "RAC2P"     "RAC3P"     "RACAIAN"   "RACASN"   
 [97] "RACBLK"    "RACNHPI"   "RACNUM"    "RACSOR"    "RACWHT"    "RC"       
[103] "SFN"       "SFR"       "SOCP"      "VPS"       "WAOB"      "FAGEP"    
[109] "FANCP"     "FCITP"     "FCOWP"     "FDDRSP"    "FDEYEP"    "FDOUTP"   
[115] "FDPHYP"    "FDREMP"    "FDWRKP"    "FENGP"     "FESRP"     "FFERP"    
[121] "FGCLP"     "FGCMP"     "FGCRP"     "FHISP"     "FINDP"     "FINTP"    
[127] "FJWDP"     "FJWMNP"    "FJWRIP"    "FJWTRP"    "FLANP"     "FLANXP"   
[133] "FMARP"     "FMIGP"     "FMIGSP"    "FMILPP"    "FMILSP"    "FMILYP"   
[139] "FOCCP"     "FOIP"      "FPAP"      "FPOBP"     "FPOWSP"    "FRACP"    
[145] "FRELP"     "FRETP"     "FSCHGP"    "FSCHLP"    "FSCHP"     "FSEMP"    
[151] "FSEXP"     "FSSIP"     "FSSP"      "FWAGP"     "FWKHP"     "FWKLP"    
[157] "FWKWP"     "FYOEP"     "PWGTP1"    "PWGTP2"    "PWGTP3"    "PWGTP4"   
[163] "PWGTP5"    "PWGTP6"    "PWGTP7"    "PWGTP8"    "PWGTP9"    "PWGTP10"  
[169] "PWGTP11"   "PWGTP12"   "PWGTP13"   "PWGTP14"   "PWGTP15"   "PWGTP16"  
[175] "PWGTP17"   "PWGTP18"   "PWGTP19"   "PWGTP20"   "PWGTP21"   "PWGTP22"  
[181] "PWGTP23"   "PWGTP24"   "PWGTP25"   "PWGTP26"   "PWGTP27"   "PWGTP28"  
[187] "PWGTP29"   "PWGTP30"   "PWGTP31"   "PWGTP32"   "PWGTP33"   "PWGTP34"  
[193] "PWGTP35"   "PWGTP36"   "PWGTP37"   "PWGTP38"   "PWGTP39"   "PWGTP40"  
[199] "PWGTP41"   "PWGTP42"   "PWGTP43"   "PWGTP44"   "PWGTP45"   "PWGTP46"  
[205] "PWGTP47"   "PWGTP48"   "PWGTP49"   "PWGTP50"   "PWGTP51"   "PWGTP52"  
[211] "PWGTP53"   "PWGTP54"   "PWGTP55"   "PWGTP56"   "PWGTP57"   "PWGTP58"  
[217] "PWGTP59"   "PWGTP60"   "PWGTP61"   "PWGTP62"   "PWGTP63"   "PWGTP64"  
[223] "PWGTP65"   "PWGTP66"   "PWGTP67"   "PWGTP68"   "PWGTP69"   "PWGTP70"  
[229] "PWGTP71"   "PWGTP72"   "PWGTP73"   "PWGTP74"   "PWGTP75"   "PWGTP76"  
[235] "PWGTP77"   "PWGTP78"   "PWGTP79"   "PWGTP80"   "NOP"       "ADJINC"   
[241] "CITWP"     "DEAR"      "DRAT"      "DRATX"     "HINS1"     "HINS2"    
[247] "HINS3"     "HINS4"     "HINS5"     "HINS6"     "HINS7"     "MARHD"    
[253] "MARHM"     "MARHT"     "MARHW"     "MARHYP"    "DIS"       "HICOV"    
[259] "PRIVCOV"   "PUBCOV"    "FCITWP"    "FDEARP"    "FDRATP"    "FDRATXP"  
[265] "FHINS1P"   "FHINS2P"   "FHINS3P"   "FHINS4P"   "FHINS5P"   "FHINS6P"  
[271] "FHINS7P"   "FMARHDP"   "FMARHMP"   "FMARHTP"   "FMARHWP"   "FMARHYP"  
[277] "WRK"       "FOD1P"     "FOD2P"     "SCIENGP"   "SCIENGRLP" "FFODP"    
[283] "FHINS3C"   "FHINS4C"   "FHINS5C"   "RELP"      "FWRKP"     "FDISP"    
[289] "FPERNP"    "FPINCP"    "FPRIVCOVP" "FPUBCOVP"  "RACNH"     "RACPI"    
[295] "SSPA"      "MLPCD"     "MLPFG"     "FHICOVP"   "DIVISION"  "REGION"   
[301] "HIMRKS"    "JWTRNS"    "RELSHIPP"  "WKWN"      "FHIMRKSP"  "FJWTRNSP" 
[307] "FRELSHIPP" "FWKWNP"    "MLPIK"     "year"      "location"

1. Concise Syntax

“why waste time say lot word when few word do trick.”


1. Concise Syntax

Fast to type, fast to read

# Base R approach
pums_df <- as.data.frame(pums)
subset_data <- pums_df[pums$ST == "California/CA" & pums$year == 2022, ]
age_groups <- ifelse(subset_data$AGEP < 18, "Under 18",
                    ifelse(subset_data$AGEP < 65, "18-64", "65+"))
base_result <- aggregate(subset_data$PWGTP, by = list(age_group = age_groups), FUN = sum)

# data.table approach
pums[, age_groups := fcase(AGEP < 18, "Under 18", AGEP < 65, "18-64", default = "65+")]
dt_result <- pums[
  year == 2022 & ST == "California/CA", 
  .(total_pop = sum(PWGTP)), 
  by = age_groups
]
base_result
  age_group        x
1     18-64 24372342
2       65+  6165865
3  Under 18  8491135
dt_result
   age_groups total_pop
       <char>     <int>
1:      18-64  24372342
2:        65+   6165865
3:   Under 18   8491135

Fill in the Blank

We want to take the pums data.table and select the SCHL, SEX, ST, and AGEP.

dt_subset <- pums[,

]

Fill in the Blank

We want to take the pums data.table and select the SCHL, SEX, ST, and AGEP.

dt_subset <- pums[,
  .(SCHL, SEX, ST, AGEP)
]

Fill in the Blank

We want to subset to those in “ny” based on our location variable, group by age_groups and year, and get the proportion of those in the “Civilian employed, at work” based on the ESR variable.

dt_ny <- pums[



]

Fill in the Blank

We want to subset to those in “ny” based on our location variable, group by age_groups and year, and get the proportion of those in the “Civilian employed, at work” based on the ESR variable.

dt_ny <- pums[
  location == "ny",
  .(employed = mean(ESR == "Civilian employed, at work", na.rm = TRUE)),
  by = .(age_groups, year)
]

What will this do?

HINSx are insurance types WAGP income (wages/salary) last 12 months

pums[, ins_type := fcase(
  HINS1 == TRUE, "employer", 
  HINS2 == TRUE, "direct", 
  HINS3 == TRUE, "Medicare", 
  HINS4 == TRUE, "Medicaid",
  HINS5 == TRUE | HINS6 == TRUE | HINS7 == TRUE, "other",
  default = "no insurance")
]
pums[, .(avg_income = mean(WAGP, na.rm = TRUE)), by = ins_type]

What will this do?

HINSx are insurance types WAGP income (wages/salary) last 12 months

pums[, ins_type := fcase(
  HINS1 == TRUE, "employer", 
  HINS2 == TRUE, "direct", 
  HINS3 == TRUE, "Medicare", 
  HINS4 == TRUE, "Medicaid",
  HINS5 == TRUE | HINS6 == TRUE | HINS7 == TRUE, "other",
  default = "no insurance")
]
pums[, .(avg_income = mean(WAGP, na.rm = TRUE)), by = ins_type]
       ins_type avg_income
         <char>      <num>
1:     Medicaid  10794.296
2:     Medicare   6044.857
3:     employer  57418.563
4:       direct  20805.087
5:        other  30429.201
6: no insurance  18047.548

2. Fast Speed

“I am faster than 80 percent of all snakes.”

Benchmarking is difficult

# Benchmarking aggregation operations
microbenchmark::microbenchmark(
  # data.table
  pums[, .(total_pop = sum(PWGTP)), by = .(ST, year)],
  
  # dplyr
  pums_df |> 
    group_by(ST, year) |> 
    summarise(total_pop = sum(PWGTP), .groups = "drop"),

  times = 10
)
Unit: milliseconds
                                                                                  expr
                                   pums[, .(total_pop = sum(PWGTP)), by = .(ST, year)]
 summarise(group_by(pums_df, ST, year), total_pop = sum(PWGTP),      .groups = "drop")
      min       lq     mean   median       uq      max neval
 11.02416 11.16004 11.37830 11.24712 11.59714 11.86429    10
 20.28008 22.07579 23.57717 23.07908 24.71168 30.27604    10

2. Fast Speed

# Benchmarking aggregation operations
microbenchmark::microbenchmark(
  # data.table
  pums[, .(total_pop = sum(PWGTP)), keyby = .(ST, year)],
  
  # dplyr
  pums_df |> 
    group_by(ST, year) |> 
    summarise(total_pop = sum(PWGTP), .groups = "drop") |> 
    arrange(ST, year),

  times = 10
)
Unit: milliseconds
                                                                                                     expr
                                                   pums[, .(total_pop = sum(PWGTP)), keyby = .(ST, year)]
 arrange(summarise(group_by(pums_df, ST, year), total_pop = sum(PWGTP),      .groups = "drop"), ST, year)
      min       lq     mean   median       uq      max neval
 35.54540 36.54670 37.06849 36.97062 37.77396 38.59371    10
 21.60811 21.90253 24.04276 23.59443 26.00901 26.99272    10

3. Memory Efficient

“Whenever I’m about to do something, I think, “would an idiot do that?” And if they would, I do not do that thing.”


Quick History Lesson on Shallow/Deep Copies

# Create a copy (memory inefficient)
pums_nocopy <- pums

# Modify by reference (memory efficient)
pums[, age_decade := floor(AGEP / 10) * 10]

# shows up in pums_nocopy
pums_nocopy[, .(age_decade)]
         age_decade
              <num>
      1:         50
      2:         40
      3:         10
      4:         50
      5:         80
     ---           
1735551:         50
1735552:         60
1735553:         20
1735554:         60
1735555:         60
lobstr::ref(pums)
█ [1:0x111cdb600] <dt[,314]> 
├─SPORDER = [2:0x1189a0000] <int> 
├─RT = [3:0x1196e0000] <fct> 
├─SERIALNO = [4:0x115900000] <chr> 
├─PUMA = [5:0x123000000] <chr> 
├─ST = [6:0x124a80000] <chr> 
├─ADJUST = [7:0x13f138000] <int> 
├─PWGTP = [8:0x328008000] <int> 
├─AGEP = [9:0x328d48000] <int> 
├─CIT = [10:0x329a88000] <fct> 
├─COW = [11:0x32a7c8000] <fct> 
├─DDRS = [12:0x1389d0000] <lgl> 
├─DEYE = [13:0x160018000] <lgl> 
├─DOUT = [14:0x178038000] <lgl> 
├─DPHY = [15:0x1786d8000] <lgl> 
├─DREM = [16:0x178d78000] <lgl> 
├─DWRK = [17:0x128aa0000] <lgl> 
├─ENG = [18:0x32b508000] <fct> 
├─FER = [19:0x129140000] <lgl> 
├─GCL = [20:0x139070000] <lgl> 
├─GCM = [21:0x32c248000] <fct> 
├─GCR = [22:0x1498d8000] <lgl> 
├─INTP = [23:0x32cf88000] <int> 
├─JWMNP = [24:0x32dcc8000] <int> 
├─JWRIP = [25:0x126500000] <chr> 
├─JWTR = [26:0x133800000] <chr> 
├─LANX = [27:0x149f78000] <lgl> 
├─MAR = [28:0x32ea08000] <fct> 
├─MIG = [29:0x14a618000] <lgl> 
├─MIL = [30:0x32f748000] <fct> 
├─MILY = [31:0x3306a8000] <fct> 
├─MLPA = [32:0x310008000] <lgl> 
├─MLPB = [33:0x1297e0000] <lgl> 
├─MLPC = [34:0x139710000] <lgl> 
├─MLPD = [35:0x139db0000] <lgl> 
├─MLPE = [36:0x300018000] <lgl> 
├─MLPF = [37:0x3006b8000] <lgl> 
├─MLPG = [38:0x300d58000] <lgl> 
├─MLPH = [39:0x14acb8000] <lgl> 
├─MLPI = [40:0x13a450000] <lgl> 
├─MLPJ = [41:0x13aaf0000] <lgl> 
├─MLPK = [42:0x13b190000] <lgl> 
├─NWAB = [43:0x13b830000] <lgl> 
├─NWAV = [44:0x3313e8000] <fct> 
├─NWLA = [45:0x13bed0000] <lgl> 
├─NWLK = [46:0x13c570000] <lgl> 
├─NWRE = [47:0x1582f8000] <lgl> 
├─OIP = [48:0x332128000] <int> 
├─PAP = [49:0x332e68000] <int> 
├─REL = [50:0x135280000] <chr> 
├─RETP = [51:0x152f70000] <int> 
├─SCH = [52:0x129e80000] <lgl> 
├─SCHG = [53:0x136d00000] <chr> 
├─SCHL = [54:0x168000000] <chr> 
├─SEMP = [55:0x153cb0000] <int> 
├─SEX = [56:0x1549f0000] <fct> 
├─SSIP = [57:0x155730000] <int> 
├─SSP = [58:0x156470000] <int> 
├─WAGP = [59:0x1571b0000] <int> 
├─WKHP = [60:0x338008000] <int> 
├─WKL = [61:0x338d48000] <fct> 
├─WKW = [62:0x339a88000] <fct> 
├─YOEP = [63:0x16f08c000] <chr> 
├─UWRK = [64:0x12a520000] <lgl> 
├─ANC = [65:0x33a7c8000] <fct> 
├─ANC1P = [66:0x340d40000] <chr> 
├─ANC2P = [67:0x3427c0000] <chr> 
├─DECADE = [68:0x11a528000] <fct> 
├─DRIVESP = [69:0x11b268000] <fct> 
├─DS = [70:0x179418000] <lgl> 
├─ESP = [71:0x11bfa8000] <fct> 
├─ESR = [72:0x11cce8000] <fct> 
├─HISP = [73:0x344240000] <chr> 
├─INDP = [74:0x345cc0000] <chr> 
├─JWAP = [75:0x347740000] <chr> 
├─JWDP = [76:0x3491c0000] <chr> 
├─LANP = [77:0x34ac40000] <chr> 
├─MIGPUMA = [78:0x12f008000] <fct> 
├─MIGSP = [79:0x34c6c0000] <chr> 
├─MSP = [80:0x33b508000] <fct> 
├─NAICSP = [81:0x34e140000] <chr> 
├─NATIVITY = [82:0x14cdd8000] <fct> 
├─OC = [83:0x150110000] <lgl> 
├─OCCP = [84:0x34fbc0000] <chr> 
├─PAOC = [85:0x358008000] <fct> 
├─PERNP = [86:0x358d48000] <int> 
├─PINCP = [87:0x359a88000] <int> 
├─POBP = [88:0x351640000] <chr> 
├─POVPIP = [89:0x35a7c8000] <int> 
├─POWPUMA = [90:0x35b508000] <fct> 
├─POWSP = [91:0x3530c0000] <chr> 
├─QTRBIR = [92:0x35c248000] <fct> 
├─RAC1P = [93:0x35cf88000] <fct> 
├─RAC2P = [94:0x354b40000] <chr> 
├─RAC3P = [95:0x3565c0000] <chr> 
├─RACAIAN = [96:0x1507b0000] <lgl> 
├─RACASN = [97:0x150e50000] <lgl> 
├─RACBLK = [98:0x1514f0000] <lgl> 
├─RACNHPI = [99:0x12abc0000] <lgl> 
├─RACNUM = [100:0x14db18000] <int> 
├─RACSOR = [101:0x13cc10000] <lgl> 
├─RACWHT = [102:0x151b90000] <lgl> 
├─RC = [103:0x1606b8000] <lgl> 
├─SFN = [104:0x14e858000] <fct> 
├─SFR = [105:0x14f598000] <fct> 
├─SOCP = [106:0x360d40000] <chr> 
├─VPS = [107:0x3627c0000] <chr> 
├─WAOB = [108:0x35dcc8000] <fct> 
├─FAGEP = [109:0x3013f8000] <lgl> 
├─FANCP = [110:0x3106a8000] <lgl> 
├─FCITP = [111:0x13d2b0000] <lgl> 
├─FCOWP = [112:0x310d48000] <lgl> 
├─FDDRSP = [113:0x3113e8000] <lgl> 
├─FDEYEP = [114:0x179ab8000] <lgl> 
├─FDOUTP = [115:0x301a98000] <lgl> 
├─FDPHYP = [116:0x158998000] <lgl> 
├─FDREMP = [117:0x311a88000] <lgl> 
├─FDWRKP = [118:0x302138000] <lgl> 
├─FENGP = [119:0x308068000] <lgl> 
├─FESRP = [120:0x3027d8000] <lgl> 
├─FFERP = [121:0x308708000] <lgl> 
├─FGCLP = [122:0x12b260000] <lgl> 
├─FGCMP = [123:0x302e78000] <lgl> 
├─FGCRP = [124:0x312128000] <lgl> 
├─FHISP = [125:0x3127c8000] <lgl> 
├─FINDP = [126:0x303518000] <lgl> 
├─FINTP = [127:0x303bb8000] <lgl> 
├─FJWDP = [128:0x12b900000] <lgl> 
├─FJWMNP = [129:0x312e68000] <lgl> 
├─FJWRIP = [130:0x152230000] <lgl> 
├─FJWTRP = [131:0x1528d0000] <lgl> 
├─FLANP = [132:0x313508000] <lgl> 
├─FLANXP = [133:0x313ba8000] <lgl> 
├─FMARP = [134:0x14b358000] <lgl> 
├─FMIGP = [135:0x170020000] <lgl> 
├─FMIGSP = [136:0x308da8000] <lgl> 
├─FMILPP = [137:0x12bfa0000] <lgl> 
├─FMILSP = [138:0x309448000] <lgl> 
├─FMILYP = [139:0x1706c0000] <lgl> 
├─FOCCP = [140:0x309ae8000] <lgl> 
├─FOIP = [141:0x304258000] <lgl> 
├─FPAP = [142:0x3048f8000] <lgl> 
├─FPOBP = [143:0x159038000] <lgl> 
├─FPOWSP = [144:0x12c640000] <lgl> 
├─FRACP = [145:0x30a188000] <lgl> 
├─FRELP = [146:0x30a828000] <lgl> 
├─FRETP = [147:0x304f98000] <lgl> 
├─FSCHGP = [148:0x305638000] <lgl> 
├─FSCHLP = [149:0x17a158000] <lgl> 
├─FSCHP = [150:0x17a7f8000] <lgl> 
├─FSEMP = [151:0x314248000] <lgl> 
├─FSEXP = [152:0x3148e8000] <lgl> 
├─FSSIP = [153:0x314f88000] <lgl> 
├─FSSP = [154:0x315628000] <lgl> 
├─FWAGP = [155:0x315cc8000] <lgl> 
├─FWKHP = [156:0x316368000] <lgl> 
├─FWKLP = [157:0x170d60000] <lgl> 
├─FWKWP = [158:0x171400000] <lgl> 
├─FYOEP = [159:0x30aec8000] <lgl> 
├─PWGTP1 = [160:0x35ea08000] <int> 
├─PWGTP2 = [161:0x35f748000] <int> 
├─PWGTP3 = [162:0x3706a8000] <int> 
├─PWGTP4 = [163:0x3713e8000] <int> 
├─PWGTP5 = [164:0x372128000] <int> 
├─PWGTP6 = [165:0x372e68000] <int> 
├─PWGTP7 = [166:0x373ba8000] <int> 
├─PWGTP8 = [167:0x3748e8000] <int> 
├─PWGTP9 = [168:0x375628000] <int> 
├─PWGTP10 = [169:0x376368000] <int> 
├─PWGTP11 = [170:0x3770a8000] <int> 
├─PWGTP12 = [171:0x378008000] <int> 
├─PWGTP13 = [172:0x378d48000] <int> 
├─PWGTP14 = [173:0x379a88000] <int> 
├─PWGTP15 = [174:0x37a7c8000] <int> 
├─PWGTP16 = [175:0x37b508000] <int> 
├─PWGTP17 = [176:0x37c248000] <int> 
├─PWGTP18 = [177:0x37cf88000] <int> 
├─PWGTP19 = [178:0x37dcc8000] <int> 
├─PWGTP20 = [179:0x37ea08000] <int> 
├─PWGTP21 = [180:0x37f748000] <int> 
├─PWGTP22 = [181:0x3806a8000] <int> 
├─PWGTP23 = [182:0x3813e8000] <int> 
├─PWGTP24 = [183:0x382128000] <int> 
├─PWGTP25 = [184:0x382e68000] <int> 
├─PWGTP26 = [185:0x383ba8000] <int> 
├─PWGTP27 = [186:0x3848e8000] <int> 
├─PWGTP28 = [187:0x385628000] <int> 
├─PWGTP29 = [188:0x386368000] <int> 
├─PWGTP30 = [189:0x3870a8000] <int> 
├─PWGTP31 = [190:0x388008000] <int> 
├─PWGTP32 = [191:0x388d48000] <int> 
├─PWGTP33 = [192:0x389a88000] <int> 
├─PWGTP34 = [193:0x33c248000] <int> 
├─PWGTP35 = [194:0x33cf88000] <int> 
├─PWGTP36 = [195:0x33dcc8000] <int> 
├─PWGTP37 = [196:0x33ea08000] <int> 
├─PWGTP38 = [197:0x33f748000] <int> 
├─PWGTP39 = [198:0x3906a8000] <int> 
├─PWGTP40 = [199:0x3913e8000] <int> 
├─PWGTP41 = [200:0x392128000] <int> 
├─PWGTP42 = [201:0x392e68000] <int> 
├─PWGTP43 = [202:0x393ba8000] <int> 
├─PWGTP44 = [203:0x3948e8000] <int> 
├─PWGTP45 = [204:0x395628000] <int> 
├─PWGTP46 = [205:0x396368000] <int> 
├─PWGTP47 = [206:0x3970a8000] <int> 
├─PWGTP48 = [207:0x398008000] <int> 
├─PWGTP49 = [208:0x398d48000] <int> 
├─PWGTP50 = [209:0x399a88000] <int> 
├─PWGTP51 = [210:0x39a7c8000] <int> 
├─PWGTP52 = [211:0x39b508000] <int> 
├─PWGTP53 = [212:0x39c248000] <int> 
├─PWGTP54 = [213:0x39cf88000] <int> 
├─PWGTP55 = [214:0x39dcc8000] <int> 
├─PWGTP56 = [215:0x39ea08000] <int> 
├─PWGTP57 = [216:0x11da28000] <int> 
├─PWGTP58 = [217:0x11e768000] <int> 
├─PWGTP59 = [218:0x11f4a8000] <int> 
├─PWGTP60 = [219:0x3a06a8000] <int> 
├─PWGTP61 = [220:0x3a13e8000] <int> 
├─PWGTP62 = [221:0x3a2128000] <int> 
├─PWGTP63 = [222:0x3a2e68000] <int> 
├─PWGTP64 = [223:0x3a3ba8000] <int> 
├─PWGTP65 = [224:0x3a48e8000] <int> 
├─PWGTP66 = [225:0x3a5628000] <int> 
├─PWGTP67 = [226:0x3a6368000] <int> 
├─PWGTP68 = [227:0x3a70a8000] <int> 
├─PWGTP69 = [228:0x3a8008000] <int> 
├─PWGTP70 = [229:0x3a8d48000] <int> 
├─PWGTP71 = [230:0x3a9a88000] <int> 
├─PWGTP72 = [231:0x3aa7c8000] <int> 
├─PWGTP73 = [232:0x3ab508000] <int> 
├─PWGTP74 = [233:0x3ac248000] <int> 
├─PWGTP75 = [234:0x3acf88000] <int> 
├─PWGTP76 = [235:0x3adcc8000] <int> 
├─PWGTP77 = [236:0x3aea08000] <int> 
├─PWGTP78 = [237:0x3af748000] <int> 
├─PWGTP79 = [238:0x3b06a8000] <int> 
├─PWGTP80 = [239:0x3b13e8000] <int> 
├─NOP = [240:0x3b2128000] <fct> 
├─ADJINC = [241:0x364240000] <dbl> 
├─CITWP = [242:0x365cc0000] <chr> 
├─DEAR = [243:0x13d950000] <lgl> 
├─DRAT = [244:0x3b2e68000] <fct> 
├─DRATX = [245:0x1596d8000] <lgl> 
├─HINS1 = [246:0x305cd8000] <lgl> 
├─HINS2 = [247:0x306378000] <lgl> 
├─HINS3 = [248:0x306a18000] <lgl> 
├─HINS4 = [249:0x3070b8000] <lgl> 
├─HINS5 = [250:0x307758000] <lgl> 
├─HINS6 = [251:0x318008000] <lgl> 
├─HINS7 = [252:0x3186a8000] <lgl> 
├─MARHD = [253:0x318d48000] <lgl> 
├─MARHM = [254:0x3193e8000] <lgl> 
├─MARHT = [255:0x3b3ba8000] <fct> 
├─MARHW = [256:0x319a88000] <lgl> 
├─MARHYP = [257:0x3b8000000] <chr> 
├─DIS = [258:0x17ae98000] <lgl> 
├─HICOV = [259:0x171aa0000] <lgl> 
├─PRIVCOV = [260:0x31a128000] <lgl> 
├─PUBCOV = [261:0x31a7c8000] <lgl> 
├─FCITWP = [262:0x172140000] <lgl> 
├─FDEARP = [263:0x316a08000] <lgl> 
├─FDRATP = [264:0x13dff0000] <lgl> 
├─FDRATXP = [265:0x13e690000] <lgl> 
├─FHINS1P = [266:0x14b9f8000] <lgl> 
├─FHINS2P = [267:0x14c098000] <lgl> 
├─FHINS3P = [268:0x1182c0000] <lgl> 
├─FHINS4P = [269:0x12cce0000] <lgl> 
├─FHINS5P = [270:0x3170a8000] <lgl> 
├─FHINS6P = [271:0x160d58000] <lgl> 
├─FHINS7P = [272:0x17b538000] <lgl> 
├─FMARHDP = [273:0x17bbd8000] <lgl> 
├─FMARHMP = [274:0x17c278000] <lgl> 
├─FMARHTP = [275:0x12d380000] <lgl> 
├─FMARHWP = [276:0x12da20000] <lgl> 
├─FMARHYP = [277:0x159d78000] <lgl> 
├─WRK = [278:0x17c918000] <lgl> 
├─FOD1P = [279:0x3b9a80000] <chr> 
├─FOD2P = [280:0x3bb500000] <chr> 
├─SCIENGP = [281:0x17cfb8000] <lgl> 
├─SCIENGRLP = [282:0x17d658000] <lgl> 
├─FFODP = [283:0x17dcf8000] <lgl> 
├─FHINS3C = [284:0x3b48e8000] <fct> 
├─FHINS4C = [285:0x3b5628000] <fct> 
├─FHINS5C = [286:0x3b6368000] <fct> 
├─RELP = [287:0x3bcf80000] <chr> 
├─FWRKP = [288:0x317748000] <lgl> 
├─FDISP = [289:0x320008000] <lgl> 
├─FPERNP = [290:0x30b568000] <lgl> 
├─FPINCP = [291:0x1613f8000] <lgl> 
├─FPRIVCOVP = [292:0x17e398000] <lgl> 
├─FPUBCOVP = [293:0x17ea38000] <lgl> 
├─RACNH = [294:0x1727e0000] <lgl> 
├─RACPI = [295:0x161a98000] <lgl> 
├─SSPA = [296:0x3b70a8000] <fct> 
├─MLPCD = [297:0x12e0c0000] <lgl> 
├─MLPFG = [298:0x30bc08000] <lgl> 
├─FHICOVP = [299:0x12e760000] <lgl> 
├─DIVISION = [300:0x3c0008000] <fct> 
├─REGION = [301:0x3c0d48000] <fct> 
├─HIMRKS = [302:0x3c1a88000] <fct> 
├─JWTRNS = [303:0x3bea00000] <chr> 
├─RELSHIPP = [304:0x3c8d40000] <chr> 
├─WKWN = [305:0x38a7c8000] <int> 
├─FHIMRKSP = [306:0x31ae68000] <lgl> 
├─FJWTRNSP = [307:0x14c738000] <lgl> 
├─FRELSHIPP = [308:0x3206a8000] <lgl> 
├─FWKWNP = [309:0x320d48000] <lgl> 
├─MLPIK = [310:0x3213e8000] <lgl> 
├─year = [311:0x38b508000] <int> 
├─location = [312:0x3ca7c0000] <chr> 
├─age_groups = [313:0x349f00000] <chr> 
├─ins_type = [314:0x34b980000] <chr> 
└─age_decade = [315:0x366a00000] <dbl> 
lobstr::ref(pums_nocopy)
█ [1:0x111cdb600] <dt[,314]> 
├─SPORDER = [2:0x1189a0000] <int> 
├─RT = [3:0x1196e0000] <fct> 
├─SERIALNO = [4:0x115900000] <chr> 
├─PUMA = [5:0x123000000] <chr> 
├─ST = [6:0x124a80000] <chr> 
├─ADJUST = [7:0x13f138000] <int> 
├─PWGTP = [8:0x328008000] <int> 
├─AGEP = [9:0x328d48000] <int> 
├─CIT = [10:0x329a88000] <fct> 
├─COW = [11:0x32a7c8000] <fct> 
├─DDRS = [12:0x1389d0000] <lgl> 
├─DEYE = [13:0x160018000] <lgl> 
├─DOUT = [14:0x178038000] <lgl> 
├─DPHY = [15:0x1786d8000] <lgl> 
├─DREM = [16:0x178d78000] <lgl> 
├─DWRK = [17:0x128aa0000] <lgl> 
├─ENG = [18:0x32b508000] <fct> 
├─FER = [19:0x129140000] <lgl> 
├─GCL = [20:0x139070000] <lgl> 
├─GCM = [21:0x32c248000] <fct> 
├─GCR = [22:0x1498d8000] <lgl> 
├─INTP = [23:0x32cf88000] <int> 
├─JWMNP = [24:0x32dcc8000] <int> 
├─JWRIP = [25:0x126500000] <chr> 
├─JWTR = [26:0x133800000] <chr> 
├─LANX = [27:0x149f78000] <lgl> 
├─MAR = [28:0x32ea08000] <fct> 
├─MIG = [29:0x14a618000] <lgl> 
├─MIL = [30:0x32f748000] <fct> 
├─MILY = [31:0x3306a8000] <fct> 
├─MLPA = [32:0x310008000] <lgl> 
├─MLPB = [33:0x1297e0000] <lgl> 
├─MLPC = [34:0x139710000] <lgl> 
├─MLPD = [35:0x139db0000] <lgl> 
├─MLPE = [36:0x300018000] <lgl> 
├─MLPF = [37:0x3006b8000] <lgl> 
├─MLPG = [38:0x300d58000] <lgl> 
├─MLPH = [39:0x14acb8000] <lgl> 
├─MLPI = [40:0x13a450000] <lgl> 
├─MLPJ = [41:0x13aaf0000] <lgl> 
├─MLPK = [42:0x13b190000] <lgl> 
├─NWAB = [43:0x13b830000] <lgl> 
├─NWAV = [44:0x3313e8000] <fct> 
├─NWLA = [45:0x13bed0000] <lgl> 
├─NWLK = [46:0x13c570000] <lgl> 
├─NWRE = [47:0x1582f8000] <lgl> 
├─OIP = [48:0x332128000] <int> 
├─PAP = [49:0x332e68000] <int> 
├─REL = [50:0x135280000] <chr> 
├─RETP = [51:0x152f70000] <int> 
├─SCH = [52:0x129e80000] <lgl> 
├─SCHG = [53:0x136d00000] <chr> 
├─SCHL = [54:0x168000000] <chr> 
├─SEMP = [55:0x153cb0000] <int> 
├─SEX = [56:0x1549f0000] <fct> 
├─SSIP = [57:0x155730000] <int> 
├─SSP = [58:0x156470000] <int> 
├─WAGP = [59:0x1571b0000] <int> 
├─WKHP = [60:0x338008000] <int> 
├─WKL = [61:0x338d48000] <fct> 
├─WKW = [62:0x339a88000] <fct> 
├─YOEP = [63:0x16f08c000] <chr> 
├─UWRK = [64:0x12a520000] <lgl> 
├─ANC = [65:0x33a7c8000] <fct> 
├─ANC1P = [66:0x340d40000] <chr> 
├─ANC2P = [67:0x3427c0000] <chr> 
├─DECADE = [68:0x11a528000] <fct> 
├─DRIVESP = [69:0x11b268000] <fct> 
├─DS = [70:0x179418000] <lgl> 
├─ESP = [71:0x11bfa8000] <fct> 
├─ESR = [72:0x11cce8000] <fct> 
├─HISP = [73:0x344240000] <chr> 
├─INDP = [74:0x345cc0000] <chr> 
├─JWAP = [75:0x347740000] <chr> 
├─JWDP = [76:0x3491c0000] <chr> 
├─LANP = [77:0x34ac40000] <chr> 
├─MIGPUMA = [78:0x12f008000] <fct> 
├─MIGSP = [79:0x34c6c0000] <chr> 
├─MSP = [80:0x33b508000] <fct> 
├─NAICSP = [81:0x34e140000] <chr> 
├─NATIVITY = [82:0x14cdd8000] <fct> 
├─OC = [83:0x150110000] <lgl> 
├─OCCP = [84:0x34fbc0000] <chr> 
├─PAOC = [85:0x358008000] <fct> 
├─PERNP = [86:0x358d48000] <int> 
├─PINCP = [87:0x359a88000] <int> 
├─POBP = [88:0x351640000] <chr> 
├─POVPIP = [89:0x35a7c8000] <int> 
├─POWPUMA = [90:0x35b508000] <fct> 
├─POWSP = [91:0x3530c0000] <chr> 
├─QTRBIR = [92:0x35c248000] <fct> 
├─RAC1P = [93:0x35cf88000] <fct> 
├─RAC2P = [94:0x354b40000] <chr> 
├─RAC3P = [95:0x3565c0000] <chr> 
├─RACAIAN = [96:0x1507b0000] <lgl> 
├─RACASN = [97:0x150e50000] <lgl> 
├─RACBLK = [98:0x1514f0000] <lgl> 
├─RACNHPI = [99:0x12abc0000] <lgl> 
├─RACNUM = [100:0x14db18000] <int> 
├─RACSOR = [101:0x13cc10000] <lgl> 
├─RACWHT = [102:0x151b90000] <lgl> 
├─RC = [103:0x1606b8000] <lgl> 
├─SFN = [104:0x14e858000] <fct> 
├─SFR = [105:0x14f598000] <fct> 
├─SOCP = [106:0x360d40000] <chr> 
├─VPS = [107:0x3627c0000] <chr> 
├─WAOB = [108:0x35dcc8000] <fct> 
├─FAGEP = [109:0x3013f8000] <lgl> 
├─FANCP = [110:0x3106a8000] <lgl> 
├─FCITP = [111:0x13d2b0000] <lgl> 
├─FCOWP = [112:0x310d48000] <lgl> 
├─FDDRSP = [113:0x3113e8000] <lgl> 
├─FDEYEP = [114:0x179ab8000] <lgl> 
├─FDOUTP = [115:0x301a98000] <lgl> 
├─FDPHYP = [116:0x158998000] <lgl> 
├─FDREMP = [117:0x311a88000] <lgl> 
├─FDWRKP = [118:0x302138000] <lgl> 
├─FENGP = [119:0x308068000] <lgl> 
├─FESRP = [120:0x3027d8000] <lgl> 
├─FFERP = [121:0x308708000] <lgl> 
├─FGCLP = [122:0x12b260000] <lgl> 
├─FGCMP = [123:0x302e78000] <lgl> 
├─FGCRP = [124:0x312128000] <lgl> 
├─FHISP = [125:0x3127c8000] <lgl> 
├─FINDP = [126:0x303518000] <lgl> 
├─FINTP = [127:0x303bb8000] <lgl> 
├─FJWDP = [128:0x12b900000] <lgl> 
├─FJWMNP = [129:0x312e68000] <lgl> 
├─FJWRIP = [130:0x152230000] <lgl> 
├─FJWTRP = [131:0x1528d0000] <lgl> 
├─FLANP = [132:0x313508000] <lgl> 
├─FLANXP = [133:0x313ba8000] <lgl> 
├─FMARP = [134:0x14b358000] <lgl> 
├─FMIGP = [135:0x170020000] <lgl> 
├─FMIGSP = [136:0x308da8000] <lgl> 
├─FMILPP = [137:0x12bfa0000] <lgl> 
├─FMILSP = [138:0x309448000] <lgl> 
├─FMILYP = [139:0x1706c0000] <lgl> 
├─FOCCP = [140:0x309ae8000] <lgl> 
├─FOIP = [141:0x304258000] <lgl> 
├─FPAP = [142:0x3048f8000] <lgl> 
├─FPOBP = [143:0x159038000] <lgl> 
├─FPOWSP = [144:0x12c640000] <lgl> 
├─FRACP = [145:0x30a188000] <lgl> 
├─FRELP = [146:0x30a828000] <lgl> 
├─FRETP = [147:0x304f98000] <lgl> 
├─FSCHGP = [148:0x305638000] <lgl> 
├─FSCHLP = [149:0x17a158000] <lgl> 
├─FSCHP = [150:0x17a7f8000] <lgl> 
├─FSEMP = [151:0x314248000] <lgl> 
├─FSEXP = [152:0x3148e8000] <lgl> 
├─FSSIP = [153:0x314f88000] <lgl> 
├─FSSP = [154:0x315628000] <lgl> 
├─FWAGP = [155:0x315cc8000] <lgl> 
├─FWKHP = [156:0x316368000] <lgl> 
├─FWKLP = [157:0x170d60000] <lgl> 
├─FWKWP = [158:0x171400000] <lgl> 
├─FYOEP = [159:0x30aec8000] <lgl> 
├─PWGTP1 = [160:0x35ea08000] <int> 
├─PWGTP2 = [161:0x35f748000] <int> 
├─PWGTP3 = [162:0x3706a8000] <int> 
├─PWGTP4 = [163:0x3713e8000] <int> 
├─PWGTP5 = [164:0x372128000] <int> 
├─PWGTP6 = [165:0x372e68000] <int> 
├─PWGTP7 = [166:0x373ba8000] <int> 
├─PWGTP8 = [167:0x3748e8000] <int> 
├─PWGTP9 = [168:0x375628000] <int> 
├─PWGTP10 = [169:0x376368000] <int> 
├─PWGTP11 = [170:0x3770a8000] <int> 
├─PWGTP12 = [171:0x378008000] <int> 
├─PWGTP13 = [172:0x378d48000] <int> 
├─PWGTP14 = [173:0x379a88000] <int> 
├─PWGTP15 = [174:0x37a7c8000] <int> 
├─PWGTP16 = [175:0x37b508000] <int> 
├─PWGTP17 = [176:0x37c248000] <int> 
├─PWGTP18 = [177:0x37cf88000] <int> 
├─PWGTP19 = [178:0x37dcc8000] <int> 
├─PWGTP20 = [179:0x37ea08000] <int> 
├─PWGTP21 = [180:0x37f748000] <int> 
├─PWGTP22 = [181:0x3806a8000] <int> 
├─PWGTP23 = [182:0x3813e8000] <int> 
├─PWGTP24 = [183:0x382128000] <int> 
├─PWGTP25 = [184:0x382e68000] <int> 
├─PWGTP26 = [185:0x383ba8000] <int> 
├─PWGTP27 = [186:0x3848e8000] <int> 
├─PWGTP28 = [187:0x385628000] <int> 
├─PWGTP29 = [188:0x386368000] <int> 
├─PWGTP30 = [189:0x3870a8000] <int> 
├─PWGTP31 = [190:0x388008000] <int> 
├─PWGTP32 = [191:0x388d48000] <int> 
├─PWGTP33 = [192:0x389a88000] <int> 
├─PWGTP34 = [193:0x33c248000] <int> 
├─PWGTP35 = [194:0x33cf88000] <int> 
├─PWGTP36 = [195:0x33dcc8000] <int> 
├─PWGTP37 = [196:0x33ea08000] <int> 
├─PWGTP38 = [197:0x33f748000] <int> 
├─PWGTP39 = [198:0x3906a8000] <int> 
├─PWGTP40 = [199:0x3913e8000] <int> 
├─PWGTP41 = [200:0x392128000] <int> 
├─PWGTP42 = [201:0x392e68000] <int> 
├─PWGTP43 = [202:0x393ba8000] <int> 
├─PWGTP44 = [203:0x3948e8000] <int> 
├─PWGTP45 = [204:0x395628000] <int> 
├─PWGTP46 = [205:0x396368000] <int> 
├─PWGTP47 = [206:0x3970a8000] <int> 
├─PWGTP48 = [207:0x398008000] <int> 
├─PWGTP49 = [208:0x398d48000] <int> 
├─PWGTP50 = [209:0x399a88000] <int> 
├─PWGTP51 = [210:0x39a7c8000] <int> 
├─PWGTP52 = [211:0x39b508000] <int> 
├─PWGTP53 = [212:0x39c248000] <int> 
├─PWGTP54 = [213:0x39cf88000] <int> 
├─PWGTP55 = [214:0x39dcc8000] <int> 
├─PWGTP56 = [215:0x39ea08000] <int> 
├─PWGTP57 = [216:0x11da28000] <int> 
├─PWGTP58 = [217:0x11e768000] <int> 
├─PWGTP59 = [218:0x11f4a8000] <int> 
├─PWGTP60 = [219:0x3a06a8000] <int> 
├─PWGTP61 = [220:0x3a13e8000] <int> 
├─PWGTP62 = [221:0x3a2128000] <int> 
├─PWGTP63 = [222:0x3a2e68000] <int> 
├─PWGTP64 = [223:0x3a3ba8000] <int> 
├─PWGTP65 = [224:0x3a48e8000] <int> 
├─PWGTP66 = [225:0x3a5628000] <int> 
├─PWGTP67 = [226:0x3a6368000] <int> 
├─PWGTP68 = [227:0x3a70a8000] <int> 
├─PWGTP69 = [228:0x3a8008000] <int> 
├─PWGTP70 = [229:0x3a8d48000] <int> 
├─PWGTP71 = [230:0x3a9a88000] <int> 
├─PWGTP72 = [231:0x3aa7c8000] <int> 
├─PWGTP73 = [232:0x3ab508000] <int> 
├─PWGTP74 = [233:0x3ac248000] <int> 
├─PWGTP75 = [234:0x3acf88000] <int> 
├─PWGTP76 = [235:0x3adcc8000] <int> 
├─PWGTP77 = [236:0x3aea08000] <int> 
├─PWGTP78 = [237:0x3af748000] <int> 
├─PWGTP79 = [238:0x3b06a8000] <int> 
├─PWGTP80 = [239:0x3b13e8000] <int> 
├─NOP = [240:0x3b2128000] <fct> 
├─ADJINC = [241:0x364240000] <dbl> 
├─CITWP = [242:0x365cc0000] <chr> 
├─DEAR = [243:0x13d950000] <lgl> 
├─DRAT = [244:0x3b2e68000] <fct> 
├─DRATX = [245:0x1596d8000] <lgl> 
├─HINS1 = [246:0x305cd8000] <lgl> 
├─HINS2 = [247:0x306378000] <lgl> 
├─HINS3 = [248:0x306a18000] <lgl> 
├─HINS4 = [249:0x3070b8000] <lgl> 
├─HINS5 = [250:0x307758000] <lgl> 
├─HINS6 = [251:0x318008000] <lgl> 
├─HINS7 = [252:0x3186a8000] <lgl> 
├─MARHD = [253:0x318d48000] <lgl> 
├─MARHM = [254:0x3193e8000] <lgl> 
├─MARHT = [255:0x3b3ba8000] <fct> 
├─MARHW = [256:0x319a88000] <lgl> 
├─MARHYP = [257:0x3b8000000] <chr> 
├─DIS = [258:0x17ae98000] <lgl> 
├─HICOV = [259:0x171aa0000] <lgl> 
├─PRIVCOV = [260:0x31a128000] <lgl> 
├─PUBCOV = [261:0x31a7c8000] <lgl> 
├─FCITWP = [262:0x172140000] <lgl> 
├─FDEARP = [263:0x316a08000] <lgl> 
├─FDRATP = [264:0x13dff0000] <lgl> 
├─FDRATXP = [265:0x13e690000] <lgl> 
├─FHINS1P = [266:0x14b9f8000] <lgl> 
├─FHINS2P = [267:0x14c098000] <lgl> 
├─FHINS3P = [268:0x1182c0000] <lgl> 
├─FHINS4P = [269:0x12cce0000] <lgl> 
├─FHINS5P = [270:0x3170a8000] <lgl> 
├─FHINS6P = [271:0x160d58000] <lgl> 
├─FHINS7P = [272:0x17b538000] <lgl> 
├─FMARHDP = [273:0x17bbd8000] <lgl> 
├─FMARHMP = [274:0x17c278000] <lgl> 
├─FMARHTP = [275:0x12d380000] <lgl> 
├─FMARHWP = [276:0x12da20000] <lgl> 
├─FMARHYP = [277:0x159d78000] <lgl> 
├─WRK = [278:0x17c918000] <lgl> 
├─FOD1P = [279:0x3b9a80000] <chr> 
├─FOD2P = [280:0x3bb500000] <chr> 
├─SCIENGP = [281:0x17cfb8000] <lgl> 
├─SCIENGRLP = [282:0x17d658000] <lgl> 
├─FFODP = [283:0x17dcf8000] <lgl> 
├─FHINS3C = [284:0x3b48e8000] <fct> 
├─FHINS4C = [285:0x3b5628000] <fct> 
├─FHINS5C = [286:0x3b6368000] <fct> 
├─RELP = [287:0x3bcf80000] <chr> 
├─FWRKP = [288:0x317748000] <lgl> 
├─FDISP = [289:0x320008000] <lgl> 
├─FPERNP = [290:0x30b568000] <lgl> 
├─FPINCP = [291:0x1613f8000] <lgl> 
├─FPRIVCOVP = [292:0x17e398000] <lgl> 
├─FPUBCOVP = [293:0x17ea38000] <lgl> 
├─RACNH = [294:0x1727e0000] <lgl> 
├─RACPI = [295:0x161a98000] <lgl> 
├─SSPA = [296:0x3b70a8000] <fct> 
├─MLPCD = [297:0x12e0c0000] <lgl> 
├─MLPFG = [298:0x30bc08000] <lgl> 
├─FHICOVP = [299:0x12e760000] <lgl> 
├─DIVISION = [300:0x3c0008000] <fct> 
├─REGION = [301:0x3c0d48000] <fct> 
├─HIMRKS = [302:0x3c1a88000] <fct> 
├─JWTRNS = [303:0x3bea00000] <chr> 
├─RELSHIPP = [304:0x3c8d40000] <chr> 
├─WKWN = [305:0x38a7c8000] <int> 
├─FHIMRKSP = [306:0x31ae68000] <lgl> 
├─FJWTRNSP = [307:0x14c738000] <lgl> 
├─FRELSHIPP = [308:0x3206a8000] <lgl> 
├─FWKWNP = [309:0x320d48000] <lgl> 
├─MLPIK = [310:0x3213e8000] <lgl> 
├─year = [311:0x38b508000] <int> 
├─location = [312:0x3ca7c0000] <chr> 
├─age_groups = [313:0x349f00000] <chr> 
├─ins_type = [314:0x34b980000] <chr> 
└─age_decade = [315:0x366a00000] <dbl>

3. Memory Efficient

Modify by reference - no copies

# Deep Copy (full dataset copied)
pums_copy <- copy(pums)

# adding variable to this does not alter the original
pums_copy[, age_decade := floor(AGEP / 10) * 10]
lobstr::ref(pums)
█ [1:0x111cdb600] <dt[,314]> 
├─SPORDER = [2:0x1189a0000] <int> 
├─RT = [3:0x1196e0000] <fct> 
├─SERIALNO = [4:0x115900000] <chr> 
├─PUMA = [5:0x123000000] <chr> 
├─ST = [6:0x124a80000] <chr> 
├─ADJUST = [7:0x13f138000] <int> 
├─PWGTP = [8:0x328008000] <int> 
├─AGEP = [9:0x328d48000] <int> 
├─CIT = [10:0x329a88000] <fct> 
├─COW = [11:0x32a7c8000] <fct> 
├─DDRS = [12:0x1389d0000] <lgl> 
├─DEYE = [13:0x160018000] <lgl> 
├─DOUT = [14:0x178038000] <lgl> 
├─DPHY = [15:0x1786d8000] <lgl> 
├─DREM = [16:0x178d78000] <lgl> 
├─DWRK = [17:0x128aa0000] <lgl> 
├─ENG = [18:0x32b508000] <fct> 
├─FER = [19:0x129140000] <lgl> 
├─GCL = [20:0x139070000] <lgl> 
├─GCM = [21:0x32c248000] <fct> 
├─GCR = [22:0x1498d8000] <lgl> 
├─INTP = [23:0x32cf88000] <int> 
├─JWMNP = [24:0x32dcc8000] <int> 
├─JWRIP = [25:0x126500000] <chr> 
├─JWTR = [26:0x133800000] <chr> 
├─LANX = [27:0x149f78000] <lgl> 
├─MAR = [28:0x32ea08000] <fct> 
├─MIG = [29:0x14a618000] <lgl> 
├─MIL = [30:0x32f748000] <fct> 
├─MILY = [31:0x3306a8000] <fct> 
├─MLPA = [32:0x310008000] <lgl> 
├─MLPB = [33:0x1297e0000] <lgl> 
├─MLPC = [34:0x139710000] <lgl> 
├─MLPD = [35:0x139db0000] <lgl> 
├─MLPE = [36:0x300018000] <lgl> 
├─MLPF = [37:0x3006b8000] <lgl> 
├─MLPG = [38:0x300d58000] <lgl> 
├─MLPH = [39:0x14acb8000] <lgl> 
├─MLPI = [40:0x13a450000] <lgl> 
├─MLPJ = [41:0x13aaf0000] <lgl> 
├─MLPK = [42:0x13b190000] <lgl> 
├─NWAB = [43:0x13b830000] <lgl> 
├─NWAV = [44:0x3313e8000] <fct> 
├─NWLA = [45:0x13bed0000] <lgl> 
├─NWLK = [46:0x13c570000] <lgl> 
├─NWRE = [47:0x1582f8000] <lgl> 
├─OIP = [48:0x332128000] <int> 
├─PAP = [49:0x332e68000] <int> 
├─REL = [50:0x135280000] <chr> 
├─RETP = [51:0x152f70000] <int> 
├─SCH = [52:0x129e80000] <lgl> 
├─SCHG = [53:0x136d00000] <chr> 
├─SCHL = [54:0x168000000] <chr> 
├─SEMP = [55:0x153cb0000] <int> 
├─SEX = [56:0x1549f0000] <fct> 
├─SSIP = [57:0x155730000] <int> 
├─SSP = [58:0x156470000] <int> 
├─WAGP = [59:0x1571b0000] <int> 
├─WKHP = [60:0x338008000] <int> 
├─WKL = [61:0x338d48000] <fct> 
├─WKW = [62:0x339a88000] <fct> 
├─YOEP = [63:0x16f08c000] <chr> 
├─UWRK = [64:0x12a520000] <lgl> 
├─ANC = [65:0x33a7c8000] <fct> 
├─ANC1P = [66:0x340d40000] <chr> 
├─ANC2P = [67:0x3427c0000] <chr> 
├─DECADE = [68:0x11a528000] <fct> 
├─DRIVESP = [69:0x11b268000] <fct> 
├─DS = [70:0x179418000] <lgl> 
├─ESP = [71:0x11bfa8000] <fct> 
├─ESR = [72:0x11cce8000] <fct> 
├─HISP = [73:0x344240000] <chr> 
├─INDP = [74:0x345cc0000] <chr> 
├─JWAP = [75:0x347740000] <chr> 
├─JWDP = [76:0x3491c0000] <chr> 
├─LANP = [77:0x34ac40000] <chr> 
├─MIGPUMA = [78:0x12f008000] <fct> 
├─MIGSP = [79:0x34c6c0000] <chr> 
├─MSP = [80:0x33b508000] <fct> 
├─NAICSP = [81:0x34e140000] <chr> 
├─NATIVITY = [82:0x14cdd8000] <fct> 
├─OC = [83:0x150110000] <lgl> 
├─OCCP = [84:0x34fbc0000] <chr> 
├─PAOC = [85:0x358008000] <fct> 
├─PERNP = [86:0x358d48000] <int> 
├─PINCP = [87:0x359a88000] <int> 
├─POBP = [88:0x351640000] <chr> 
├─POVPIP = [89:0x35a7c8000] <int> 
├─POWPUMA = [90:0x35b508000] <fct> 
├─POWSP = [91:0x3530c0000] <chr> 
├─QTRBIR = [92:0x35c248000] <fct> 
├─RAC1P = [93:0x35cf88000] <fct> 
├─RAC2P = [94:0x354b40000] <chr> 
├─RAC3P = [95:0x3565c0000] <chr> 
├─RACAIAN = [96:0x1507b0000] <lgl> 
├─RACASN = [97:0x150e50000] <lgl> 
├─RACBLK = [98:0x1514f0000] <lgl> 
├─RACNHPI = [99:0x12abc0000] <lgl> 
├─RACNUM = [100:0x14db18000] <int> 
├─RACSOR = [101:0x13cc10000] <lgl> 
├─RACWHT = [102:0x151b90000] <lgl> 
├─RC = [103:0x1606b8000] <lgl> 
├─SFN = [104:0x14e858000] <fct> 
├─SFR = [105:0x14f598000] <fct> 
├─SOCP = [106:0x360d40000] <chr> 
├─VPS = [107:0x3627c0000] <chr> 
├─WAOB = [108:0x35dcc8000] <fct> 
├─FAGEP = [109:0x3013f8000] <lgl> 
├─FANCP = [110:0x3106a8000] <lgl> 
├─FCITP = [111:0x13d2b0000] <lgl> 
├─FCOWP = [112:0x310d48000] <lgl> 
├─FDDRSP = [113:0x3113e8000] <lgl> 
├─FDEYEP = [114:0x179ab8000] <lgl> 
├─FDOUTP = [115:0x301a98000] <lgl> 
├─FDPHYP = [116:0x158998000] <lgl> 
├─FDREMP = [117:0x311a88000] <lgl> 
├─FDWRKP = [118:0x302138000] <lgl> 
├─FENGP = [119:0x308068000] <lgl> 
├─FESRP = [120:0x3027d8000] <lgl> 
├─FFERP = [121:0x308708000] <lgl> 
├─FGCLP = [122:0x12b260000] <lgl> 
├─FGCMP = [123:0x302e78000] <lgl> 
├─FGCRP = [124:0x312128000] <lgl> 
├─FHISP = [125:0x3127c8000] <lgl> 
├─FINDP = [126:0x303518000] <lgl> 
├─FINTP = [127:0x303bb8000] <lgl> 
├─FJWDP = [128:0x12b900000] <lgl> 
├─FJWMNP = [129:0x312e68000] <lgl> 
├─FJWRIP = [130:0x152230000] <lgl> 
├─FJWTRP = [131:0x1528d0000] <lgl> 
├─FLANP = [132:0x313508000] <lgl> 
├─FLANXP = [133:0x313ba8000] <lgl> 
├─FMARP = [134:0x14b358000] <lgl> 
├─FMIGP = [135:0x170020000] <lgl> 
├─FMIGSP = [136:0x308da8000] <lgl> 
├─FMILPP = [137:0x12bfa0000] <lgl> 
├─FMILSP = [138:0x309448000] <lgl> 
├─FMILYP = [139:0x1706c0000] <lgl> 
├─FOCCP = [140:0x309ae8000] <lgl> 
├─FOIP = [141:0x304258000] <lgl> 
├─FPAP = [142:0x3048f8000] <lgl> 
├─FPOBP = [143:0x159038000] <lgl> 
├─FPOWSP = [144:0x12c640000] <lgl> 
├─FRACP = [145:0x30a188000] <lgl> 
├─FRELP = [146:0x30a828000] <lgl> 
├─FRETP = [147:0x304f98000] <lgl> 
├─FSCHGP = [148:0x305638000] <lgl> 
├─FSCHLP = [149:0x17a158000] <lgl> 
├─FSCHP = [150:0x17a7f8000] <lgl> 
├─FSEMP = [151:0x314248000] <lgl> 
├─FSEXP = [152:0x3148e8000] <lgl> 
├─FSSIP = [153:0x314f88000] <lgl> 
├─FSSP = [154:0x315628000] <lgl> 
├─FWAGP = [155:0x315cc8000] <lgl> 
├─FWKHP = [156:0x316368000] <lgl> 
├─FWKLP = [157:0x170d60000] <lgl> 
├─FWKWP = [158:0x171400000] <lgl> 
├─FYOEP = [159:0x30aec8000] <lgl> 
├─PWGTP1 = [160:0x35ea08000] <int> 
├─PWGTP2 = [161:0x35f748000] <int> 
├─PWGTP3 = [162:0x3706a8000] <int> 
├─PWGTP4 = [163:0x3713e8000] <int> 
├─PWGTP5 = [164:0x372128000] <int> 
├─PWGTP6 = [165:0x372e68000] <int> 
├─PWGTP7 = [166:0x373ba8000] <int> 
├─PWGTP8 = [167:0x3748e8000] <int> 
├─PWGTP9 = [168:0x375628000] <int> 
├─PWGTP10 = [169:0x376368000] <int> 
├─PWGTP11 = [170:0x3770a8000] <int> 
├─PWGTP12 = [171:0x378008000] <int> 
├─PWGTP13 = [172:0x378d48000] <int> 
├─PWGTP14 = [173:0x379a88000] <int> 
├─PWGTP15 = [174:0x37a7c8000] <int> 
├─PWGTP16 = [175:0x37b508000] <int> 
├─PWGTP17 = [176:0x37c248000] <int> 
├─PWGTP18 = [177:0x37cf88000] <int> 
├─PWGTP19 = [178:0x37dcc8000] <int> 
├─PWGTP20 = [179:0x37ea08000] <int> 
├─PWGTP21 = [180:0x37f748000] <int> 
├─PWGTP22 = [181:0x3806a8000] <int> 
├─PWGTP23 = [182:0x3813e8000] <int> 
├─PWGTP24 = [183:0x382128000] <int> 
├─PWGTP25 = [184:0x382e68000] <int> 
├─PWGTP26 = [185:0x383ba8000] <int> 
├─PWGTP27 = [186:0x3848e8000] <int> 
├─PWGTP28 = [187:0x385628000] <int> 
├─PWGTP29 = [188:0x386368000] <int> 
├─PWGTP30 = [189:0x3870a8000] <int> 
├─PWGTP31 = [190:0x388008000] <int> 
├─PWGTP32 = [191:0x388d48000] <int> 
├─PWGTP33 = [192:0x389a88000] <int> 
├─PWGTP34 = [193:0x33c248000] <int> 
├─PWGTP35 = [194:0x33cf88000] <int> 
├─PWGTP36 = [195:0x33dcc8000] <int> 
├─PWGTP37 = [196:0x33ea08000] <int> 
├─PWGTP38 = [197:0x33f748000] <int> 
├─PWGTP39 = [198:0x3906a8000] <int> 
├─PWGTP40 = [199:0x3913e8000] <int> 
├─PWGTP41 = [200:0x392128000] <int> 
├─PWGTP42 = [201:0x392e68000] <int> 
├─PWGTP43 = [202:0x393ba8000] <int> 
├─PWGTP44 = [203:0x3948e8000] <int> 
├─PWGTP45 = [204:0x395628000] <int> 
├─PWGTP46 = [205:0x396368000] <int> 
├─PWGTP47 = [206:0x3970a8000] <int> 
├─PWGTP48 = [207:0x398008000] <int> 
├─PWGTP49 = [208:0x398d48000] <int> 
├─PWGTP50 = [209:0x399a88000] <int> 
├─PWGTP51 = [210:0x39a7c8000] <int> 
├─PWGTP52 = [211:0x39b508000] <int> 
├─PWGTP53 = [212:0x39c248000] <int> 
├─PWGTP54 = [213:0x39cf88000] <int> 
├─PWGTP55 = [214:0x39dcc8000] <int> 
├─PWGTP56 = [215:0x39ea08000] <int> 
├─PWGTP57 = [216:0x11da28000] <int> 
├─PWGTP58 = [217:0x11e768000] <int> 
├─PWGTP59 = [218:0x11f4a8000] <int> 
├─PWGTP60 = [219:0x3a06a8000] <int> 
├─PWGTP61 = [220:0x3a13e8000] <int> 
├─PWGTP62 = [221:0x3a2128000] <int> 
├─PWGTP63 = [222:0x3a2e68000] <int> 
├─PWGTP64 = [223:0x3a3ba8000] <int> 
├─PWGTP65 = [224:0x3a48e8000] <int> 
├─PWGTP66 = [225:0x3a5628000] <int> 
├─PWGTP67 = [226:0x3a6368000] <int> 
├─PWGTP68 = [227:0x3a70a8000] <int> 
├─PWGTP69 = [228:0x3a8008000] <int> 
├─PWGTP70 = [229:0x3a8d48000] <int> 
├─PWGTP71 = [230:0x3a9a88000] <int> 
├─PWGTP72 = [231:0x3aa7c8000] <int> 
├─PWGTP73 = [232:0x3ab508000] <int> 
├─PWGTP74 = [233:0x3ac248000] <int> 
├─PWGTP75 = [234:0x3acf88000] <int> 
├─PWGTP76 = [235:0x3adcc8000] <int> 
├─PWGTP77 = [236:0x3aea08000] <int> 
├─PWGTP78 = [237:0x3af748000] <int> 
├─PWGTP79 = [238:0x3b06a8000] <int> 
├─PWGTP80 = [239:0x3b13e8000] <int> 
├─NOP = [240:0x3b2128000] <fct> 
├─ADJINC = [241:0x364240000] <dbl> 
├─CITWP = [242:0x365cc0000] <chr> 
├─DEAR = [243:0x13d950000] <lgl> 
├─DRAT = [244:0x3b2e68000] <fct> 
├─DRATX = [245:0x1596d8000] <lgl> 
├─HINS1 = [246:0x305cd8000] <lgl> 
├─HINS2 = [247:0x306378000] <lgl> 
├─HINS3 = [248:0x306a18000] <lgl> 
├─HINS4 = [249:0x3070b8000] <lgl> 
├─HINS5 = [250:0x307758000] <lgl> 
├─HINS6 = [251:0x318008000] <lgl> 
├─HINS7 = [252:0x3186a8000] <lgl> 
├─MARHD = [253:0x318d48000] <lgl> 
├─MARHM = [254:0x3193e8000] <lgl> 
├─MARHT = [255:0x3b3ba8000] <fct> 
├─MARHW = [256:0x319a88000] <lgl> 
├─MARHYP = [257:0x3b8000000] <chr> 
├─DIS = [258:0x17ae98000] <lgl> 
├─HICOV = [259:0x171aa0000] <lgl> 
├─PRIVCOV = [260:0x31a128000] <lgl> 
├─PUBCOV = [261:0x31a7c8000] <lgl> 
├─FCITWP = [262:0x172140000] <lgl> 
├─FDEARP = [263:0x316a08000] <lgl> 
├─FDRATP = [264:0x13dff0000] <lgl> 
├─FDRATXP = [265:0x13e690000] <lgl> 
├─FHINS1P = [266:0x14b9f8000] <lgl> 
├─FHINS2P = [267:0x14c098000] <lgl> 
├─FHINS3P = [268:0x1182c0000] <lgl> 
├─FHINS4P = [269:0x12cce0000] <lgl> 
├─FHINS5P = [270:0x3170a8000] <lgl> 
├─FHINS6P = [271:0x160d58000] <lgl> 
├─FHINS7P = [272:0x17b538000] <lgl> 
├─FMARHDP = [273:0x17bbd8000] <lgl> 
├─FMARHMP = [274:0x17c278000] <lgl> 
├─FMARHTP = [275:0x12d380000] <lgl> 
├─FMARHWP = [276:0x12da20000] <lgl> 
├─FMARHYP = [277:0x159d78000] <lgl> 
├─WRK = [278:0x17c918000] <lgl> 
├─FOD1P = [279:0x3b9a80000] <chr> 
├─FOD2P = [280:0x3bb500000] <chr> 
├─SCIENGP = [281:0x17cfb8000] <lgl> 
├─SCIENGRLP = [282:0x17d658000] <lgl> 
├─FFODP = [283:0x17dcf8000] <lgl> 
├─FHINS3C = [284:0x3b48e8000] <fct> 
├─FHINS4C = [285:0x3b5628000] <fct> 
├─FHINS5C = [286:0x3b6368000] <fct> 
├─RELP = [287:0x3bcf80000] <chr> 
├─FWRKP = [288:0x317748000] <lgl> 
├─FDISP = [289:0x320008000] <lgl> 
├─FPERNP = [290:0x30b568000] <lgl> 
├─FPINCP = [291:0x1613f8000] <lgl> 
├─FPRIVCOVP = [292:0x17e398000] <lgl> 
├─FPUBCOVP = [293:0x17ea38000] <lgl> 
├─RACNH = [294:0x1727e0000] <lgl> 
├─RACPI = [295:0x161a98000] <lgl> 
├─SSPA = [296:0x3b70a8000] <fct> 
├─MLPCD = [297:0x12e0c0000] <lgl> 
├─MLPFG = [298:0x30bc08000] <lgl> 
├─FHICOVP = [299:0x12e760000] <lgl> 
├─DIVISION = [300:0x3c0008000] <fct> 
├─REGION = [301:0x3c0d48000] <fct> 
├─HIMRKS = [302:0x3c1a88000] <fct> 
├─JWTRNS = [303:0x3bea00000] <chr> 
├─RELSHIPP = [304:0x3c8d40000] <chr> 
├─WKWN = [305:0x38a7c8000] <int> 
├─FHIMRKSP = [306:0x31ae68000] <lgl> 
├─FJWTRNSP = [307:0x14c738000] <lgl> 
├─FRELSHIPP = [308:0x3206a8000] <lgl> 
├─FWKWNP = [309:0x320d48000] <lgl> 
├─MLPIK = [310:0x3213e8000] <lgl> 
├─year = [311:0x38b508000] <int> 
├─location = [312:0x3ca7c0000] <chr> 
├─age_groups = [313:0x349f00000] <chr> 
├─ins_type = [314:0x34b980000] <chr> 
└─age_decade = [315:0x366a00000] <dbl> 
lobstr::ref(pums_copy)
█ [1:0x110a2a800] <dt[,314]> 
├─SPORDER = [2:0x3c13e8000] <int> 
├─RT = [3:0x11ee08000] <fct> 
├─SERIALNO = [4:0x3ba7c0000] <chr> 
├─PUMA = [5:0x3b8d40000] <chr> 
├─ST = [6:0x34d400000] <chr> 
├─ADJUST = [7:0x11abc8000] <int> 
├─PWGTP = [8:0x11d388000] <int> 
├─AGEP = [9:0x119040000] <int> 
├─CIT = [10:0x11b908000] <fct> 
├─COW = [11:0x11c648000] <fct> 
├─DDRS = [12:0x462208000] <lgl> 
├─DEYE = [13:0x4678a8000] <lgl> 
├─DOUT = [14:0x462ba8000] <lgl> 
├─DPHY = [15:0x463ee8000] <lgl> 
├─DREM = [16:0x464888000] <lgl> 
├─DWRK = [17:0x463548000] <lgl> 
├─ENG = [18:0x15f398000] <fct> 
├─FER = [19:0x465bc8000] <lgl> 
├─GCL = [20:0x465228000] <lgl> 
├─GCM = [21:0x466568000] <fct> 
├─GCR = [22:0x466f08000] <lgl> 
├─INTP = [23:0x461868000] <int> 
├─JWMNP = [24:0x460ec8000] <int> 
├─JWRIP = [25:0x34ee80000] <chr> 
├─JWTR = [26:0x350900000] <chr> 
├─LANX = [27:0x460008000] <lgl> 
├─MAR = [28:0x4606a8000] <fct> 
├─MIG = [29:0x457388000] <lgl> 
├─MIL = [30:0x4539c8000] <fct> 
├─MILY = [31:0x450008000] <fct> 
├─MLPA = [32:0x455ec8000] <lgl> 
├─MLPB = [33:0x455528000] <lgl> 
├─MLPC = [34:0x454b88000] <lgl> 
├─MLPD = [35:0x4541e8000] <lgl> 
├─MLPE = [36:0x452508000] <lgl> 
├─MLPF = [37:0x451b68000] <lgl> 
├─MLPG = [38:0x3c2128000] <lgl> 
├─MLPH = [39:0x3c27c8000] <lgl> 
├─MLPI = [40:0x3c2e68000] <lgl> 
├─MLPJ = [41:0x3dae68000] <lgl> 
├─MLPK = [42:0x3db508000] <lgl> 
├─NWAB = [43:0x3dbba8000] <lgl> 
├─NWAV = [44:0x3dc248000] <fct> 
├─NWLA = [45:0x3dc8e8000] <lgl> 
├─NWLK = [46:0x3dcf88000] <lgl> 
├─NWRE = [47:0x3dd628000] <lgl> 
├─OIP = [48:0x3ddcc8000] <int> 
├─PAP = [49:0x3de368000] <int> 
├─REL = [50:0x352380000] <chr> 
├─RETP = [51:0x3dea08000] <int> 
├─SCH = [52:0x3df0a8000] <lgl> 
├─SCHG = [53:0x353e00000] <chr> 
├─SCHL = [54:0x355880000] <chr> 
├─SEMP = [55:0x3df748000] <int> 
├─SEX = [56:0x4778a8000] <fct> 
├─SSIP = [57:0x476f08000] <int> 
├─SSP = [58:0x476568000] <int> 
├─WAGP = [59:0x475bc8000] <int> 
├─WKHP = [60:0x4750a8000] <int> 
├─WKL = [61:0x474708000] <fct> 
├─WKW = [62:0x473d68000] <fct> 
├─YOEP = [63:0x360000000] <chr> 
├─UWRK = [64:0x4733c8000] <lgl> 
├─ANC = [65:0x472a28000] <fct> 
├─ANC1P = [66:0x361a80000] <chr> 
├─ANC2P = [67:0x363500000] <chr> 
├─DECADE = [68:0x472088000] <fct> 
├─DRIVESP = [69:0x4716e8000] <fct> 
├─DS = [70:0x470008000] <lgl> 
├─ESP = [71:0x4706a8000] <fct> 
├─ESR = [72:0x470d48000] <fct> 
├─HISP = [73:0x364f80000] <chr> 
├─INDP = [74:0x3bc240000] <chr> 
├─JWAP = [75:0x3bdcc0000] <chr> 
├─JWDP = [76:0x490808000] <chr> 
├─LANP = [77:0x492284000] <chr> 
├─MIGPUMA = [78:0x487808000] <fct> 
├─MIGSP = [79:0x493d00000] <chr> 
├─MSP = [80:0x486e68000] <fct> 
├─NAICSP = [81:0x49577c000] <chr> 
├─NATIVITY = [82:0x4864c8000] <fct> 
├─OC = [83:0x485b28000] <lgl> 
├─OCCP = [84:0x4971f8000] <chr> 
├─PAOC = [85:0x485188000] <fct> 
├─PERNP = [86:0x4847e8000] <int> 
├─PINCP = [87:0x483e48000] <int> 
├─POBP = [88:0x498c74000] <chr> 
├─POVPIP = [89:0x483328000] <int> 
├─POWPUMA = [90:0x482808000] <fct> 
├─POWSP = [91:0x49a6f0000] <chr> 
├─QTRBIR = [92:0x481e68000] <fct> 
├─RAC1P = [93:0x4814c8000] <fct> 
├─RAC2P = [94:0x49c16c000] <chr> 
├─RAC3P = [95:0x49dbe8000] <chr> 
├─RACAIAN = [96:0x480b28000] <lgl> 
├─RACASN = [97:0x480188000] <lgl> 
├─RACBLK = [98:0x46efe8000] <lgl> 
├─RACNHPI = [99:0x46e648000] <lgl> 
├─RACNUM = [100:0x46dca8000] <int> 
├─RACSOR = [101:0x46d308000] <lgl> 
├─RACWHT = [102:0x46c968000] <lgl> 
├─RC = [103:0x46be48000] <lgl> 
├─SFN = [104:0x46b328000] <fct> 
├─SFR = [105:0x46a988000] <fct> 
├─SOCP = [106:0x49f664000] <chr> 
├─VPS = [107:0x4a10e0000] <chr> 
├─WAOB = [108:0x469fe8000] <fct> 
├─FAGEP = [109:0x469648000] <lgl> 
├─FANCP = [110:0x468ca8000] <lgl> 
├─FCITP = [111:0x468308000] <lgl> 
├─FCOWP = [112:0x45f088000] <lgl> 
├─FDDRSP = [113:0x45e6e8000] <lgl> 
├─FDEYEP = [114:0x45dd48000] <lgl> 
├─FDOUTP = [115:0x45d3a8000] <lgl> 
├─FDPHYP = [116:0x45ca08000] <lgl> 
├─FDREMP = [117:0x45c068000] <lgl> 
├─FDWRKP = [118:0x45b6c8000] <lgl> 
├─FENGP = [119:0x45ad28000] <lgl> 
├─FESRP = [120:0x45a388000] <lgl> 
├─FFERP = [121:0x4599e8000] <lgl> 
├─FGCLP = [122:0x459048000] <lgl> 
├─FGCMP = [123:0x458008000] <lgl> 
├─FGCRP = [124:0x4586a8000] <lgl> 
├─FHISP = [125:0x44f388000] <lgl> 
├─FINDP = [126:0x44e9e8000] <lgl> 
├─FINTP = [127:0x44e048000] <lgl> 
├─FJWDP = [128:0x44d6a8000] <lgl> 
├─FJWMNP = [129:0x44cd08000] <lgl> 
├─FJWRIP = [130:0x44c368000] <lgl> 
├─FJWTRP = [131:0x44b9c8000] <lgl> 
├─FLANP = [132:0x44b028000] <lgl> 
├─FLANXP = [133:0x44a688000] <lgl> 
├─FMARP = [134:0x449ce8000] <lgl> 
├─FMIGP = [135:0x449348000] <lgl> 
├─FMIGSP = [136:0x4489a8000] <lgl> 
├─FMILPP = [137:0x448008000] <lgl> 
├─FMILSP = [138:0x447508000] <lgl> 
├─FMILYP = [139:0x446b68000] <lgl> 
├─FOCCP = [140:0x4461c8000] <lgl> 
├─FOIP = [141:0x445828000] <lgl> 
├─FPAP = [142:0x444e88000] <lgl> 
├─FPOBP = [143:0x4444e8000] <lgl> 
├─FPOWSP = [144:0x443b48000] <lgl> 
├─FRACP = [145:0x4431a8000] <lgl> 
├─FRELP = [146:0x442808000] <lgl> 
├─FRETP = [147:0x441e68000] <lgl> 
├─FSCHGP = [148:0x4414c8000] <lgl> 
├─FSCHLP = [149:0x440b28000] <lgl> 
├─FSCHP = [150:0x440188000] <lgl> 
├─FSEMP = [151:0x4a8008000] <lgl> 
├─FSEXP = [152:0x4a86a8000] <lgl> 
├─FSSIP = [153:0x4a8d48000] <lgl> 
├─FSSP = [154:0x4a93e8000] <lgl> 
├─FWAGP = [155:0x4a9a88000] <lgl> 
├─FWKHP = [156:0x4aa128000] <lgl> 
├─FWKLP = [157:0x4aa7c8000] <lgl> 
├─FWKWP = [158:0x4aae68000] <lgl> 
├─FYOEP = [159:0x4ab508000] <lgl> 
├─PWGTP1 = [160:0x4abba8000] <int> 
├─PWGTP2 = [161:0x4ac248000] <int> 
├─PWGTP3 = [162:0x4ac8e8000] <int> 
├─PWGTP4 = [163:0x4acf88000] <int> 
├─PWGTP5 = [164:0x4ad628000] <int> 
├─PWGTP6 = [165:0x4adcc8000] <int> 
├─PWGTP7 = [166:0x4ae368000] <int> 
├─PWGTP8 = [167:0x4aea08000] <int> 
├─PWGTP9 = [168:0x4af0a8000] <int> 
├─PWGTP10 = [169:0x4af748000] <int> 
├─PWGTP11 = [170:0x4b0008000] <int> 
├─PWGTP12 = [171:0x4b06a8000] <int> 
├─PWGTP13 = [172:0x4b0d48000] <int> 
├─PWGTP14 = [173:0x4b13e8000] <int> 
├─PWGTP15 = [174:0x4b1a88000] <int> 
├─PWGTP16 = [175:0x4b2128000] <int> 
├─PWGTP17 = [176:0x4b27c8000] <int> 
├─PWGTP18 = [177:0x4b2e68000] <int> 
├─PWGTP19 = [178:0x4b3508000] <int> 
├─PWGTP20 = [179:0x4b3ba8000] <int> 
├─PWGTP21 = [180:0x4b4248000] <int> 
├─PWGTP22 = [181:0x4b48e8000] <int> 
├─PWGTP23 = [182:0x4b4f88000] <int> 
├─PWGTP24 = [183:0x4b5628000] <int> 
├─PWGTP25 = [184:0x4b5cc8000] <int> 
├─PWGTP26 = [185:0x4b6368000] <int> 
├─PWGTP27 = [186:0x4b6a08000] <int> 
├─PWGTP28 = [187:0x4b70a8000] <int> 
├─PWGTP29 = [188:0x4b7748000] <int> 
├─PWGTP30 = [189:0x4b8008000] <int> 
├─PWGTP31 = [190:0x4b86a8000] <int> 
├─PWGTP32 = [191:0x4b8d48000] <int> 
├─PWGTP33 = [192:0x4b93e8000] <int> 
├─PWGTP34 = [193:0x4b9a88000] <int> 
├─PWGTP35 = [194:0x4ba128000] <int> 
├─PWGTP36 = [195:0x4ba7c8000] <int> 
├─PWGTP37 = [196:0x4bae68000] <int> 
├─PWGTP38 = [197:0x4bb508000] <int> 
├─PWGTP39 = [198:0x4bbba8000] <int> 
├─PWGTP40 = [199:0x4bc248000] <int> 
├─PWGTP41 = [200:0x4bc8e8000] <int> 
├─PWGTP42 = [201:0x4bcf88000] <int> 
├─PWGTP43 = [202:0x4bd628000] <int> 
├─PWGTP44 = [203:0x4bdcc8000] <int> 
├─PWGTP45 = [204:0x4be368000] <int> 
├─PWGTP46 = [205:0x4bea08000] <int> 
├─PWGTP47 = [206:0x4bf0a8000] <int> 
├─PWGTP48 = [207:0x4bf748000] <int> 
├─PWGTP49 = [208:0x4c0008000] <int> 
├─PWGTP50 = [209:0x4c06a8000] <int> 
├─PWGTP51 = [210:0x4c0d48000] <int> 
├─PWGTP52 = [211:0x4c13e8000] <int> 
├─PWGTP53 = [212:0x4c1a88000] <int> 
├─PWGTP54 = [213:0x4c2128000] <int> 
├─PWGTP55 = [214:0x4c27c8000] <int> 
├─PWGTP56 = [215:0x4c2e68000] <int> 
├─PWGTP57 = [216:0x4c3508000] <int> 
├─PWGTP58 = [217:0x4c3ba8000] <int> 
├─PWGTP59 = [218:0x4c4248000] <int> 
├─PWGTP60 = [219:0x4c48e8000] <int> 
├─PWGTP61 = [220:0x4c4f88000] <int> 
├─PWGTP62 = [221:0x4c5628000] <int> 
├─PWGTP63 = [222:0x4c5cc8000] <int> 
├─PWGTP64 = [223:0x4c6368000] <int> 
├─PWGTP65 = [224:0x4c6a08000] <int> 
├─PWGTP66 = [225:0x4c70a8000] <int> 
├─PWGTP67 = [226:0x4c7748000] <int> 
├─PWGTP68 = [227:0x4c8008000] <int> 
├─PWGTP69 = [228:0x4c86a8000] <int> 
├─PWGTP70 = [229:0x4c8d48000] <int> 
├─PWGTP71 = [230:0x4c93e8000] <int> 
├─PWGTP72 = [231:0x4c9a88000] <int> 
├─PWGTP73 = [232:0x4ca128000] <int> 
├─PWGTP74 = [233:0x4ca7c8000] <int> 
├─PWGTP75 = [234:0x4cae68000] <int> 
├─PWGTP76 = [235:0x4cb508000] <int> 
├─PWGTP77 = [236:0x4cbba8000] <int> 
├─PWGTP78 = [237:0x4cc248000] <int> 
├─PWGTP79 = [238:0x4cc8e8000] <int> 
├─PWGTP80 = [239:0x4ccf88000] <int> 
├─NOP = [240:0x4cd628000] <fct> 
├─ADJINC = [241:0x4a1e20000] <dbl> 
├─CITWP = [242:0x4a2b60000] <chr> 
├─DEAR = [243:0x4cdcc8000] <lgl> 
├─DRAT = [244:0x4ce368000] <fct> 
├─DRATX = [245:0x4cea08000] <lgl> 
├─HINS1 = [246:0x4cf0a8000] <lgl> 
├─HINS2 = [247:0x4cf748000] <lgl> 
├─HINS3 = [248:0x4d0008000] <lgl> 
├─HINS4 = [249:0x4d06a8000] <lgl> 
├─HINS5 = [250:0x4d0d48000] <lgl> 
├─HINS6 = [251:0x4d13e8000] <lgl> 
├─HINS7 = [252:0x4d1a88000] <lgl> 
├─MARHD = [253:0x4d2128000] <lgl> 
├─MARHM = [254:0x4d27c8000] <lgl> 
├─MARHT = [255:0x4d2e68000] <fct> 
├─MARHW = [256:0x4d3508000] <lgl> 
├─MARHYP = [257:0x4a38a0000] <chr> 
├─DIS = [258:0x4d3ba8000] <lgl> 
├─HICOV = [259:0x4d4248000] <lgl> 
├─PRIVCOV = [260:0x4d48e8000] <lgl> 
├─PUBCOV = [261:0x4d4f88000] <lgl> 
├─FCITWP = [262:0x4d5628000] <lgl> 
├─FDEARP = [263:0x4d5cc8000] <lgl> 
├─FDRATP = [264:0x4d6368000] <lgl> 
├─FDRATXP = [265:0x4d6a08000] <lgl> 
├─FHINS1P = [266:0x4d70a8000] <lgl> 
├─FHINS2P = [267:0x4d7748000] <lgl> 
├─FHINS3P = [268:0x4d8008000] <lgl> 
├─FHINS4P = [269:0x4d86a8000] <lgl> 
├─FHINS5P = [270:0x4d8d48000] <lgl> 
├─FHINS6P = [271:0x4d93e8000] <lgl> 
├─FHINS7P = [272:0x4d9a88000] <lgl> 
├─FMARHDP = [273:0x4da128000] <lgl> 
├─FMARHMP = [274:0x4da7c8000] <lgl> 
├─FMARHTP = [275:0x4dae68000] <lgl> 
├─FMARHWP = [276:0x4db508000] <lgl> 
├─FMARHYP = [277:0x4dbba8000] <lgl> 
├─WRK = [278:0x4dc248000] <lgl> 
├─FOD1P = [279:0x4a45e0000] <chr> 
├─FOD2P = [280:0x4a5320000] <chr> 
├─SCIENGP = [281:0x4dc8e8000] <lgl> 
├─SCIENGRLP = [282:0x4dcf88000] <lgl> 
├─FFODP = [283:0x4dd628000] <lgl> 
├─FHINS3C = [284:0x4ddcc8000] <fct> 
├─FHINS4C = [285:0x4de368000] <fct> 
├─FHINS5C = [286:0x4dea08000] <fct> 
├─RELP = [287:0x4a6060000] <chr> 
├─FWRKP = [288:0x4df0a8000] <lgl> 
├─FDISP = [289:0x4df748000] <lgl> 
├─FPERNP = [290:0x4e0008000] <lgl> 
├─FPINCP = [291:0x4e06a8000] <lgl> 
├─FPRIVCOVP = [292:0x4e0d48000] <lgl> 
├─FPUBCOVP = [293:0x4e13e8000] <lgl> 
├─RACNH = [294:0x4e1a88000] <lgl> 
├─RACPI = [295:0x4e2128000] <lgl> 
├─SSPA = [296:0x4e27c8000] <fct> 
├─MLPCD = [297:0x4e2e68000] <lgl> 
├─MLPFG = [298:0x4e3508000] <lgl> 
├─FHICOVP = [299:0x4e3ba8000] <lgl> 
├─DIVISION = [300:0x4e4248000] <fct> 
├─REGION = [301:0x4e48e8000] <fct> 
├─HIMRKS = [302:0x4e4f88000] <fct> 
├─JWTRNS = [303:0x4a6da0000] <chr> 
├─RELSHIPP = [304:0x4e8000000] <chr> 
├─WKWN = [305:0x4e5628000] <int> 
├─FHIMRKSP = [306:0x4e5cc8000] <lgl> 
├─FJWTRNSP = [307:0x4e6368000] <lgl> 
├─FRELSHIPP = [308:0x4e6a08000] <lgl> 
├─FWKWNP = [309:0x4e70a8000] <lgl> 
├─MLPIK = [310:0x4e7748000] <lgl> 
├─year = [311:0x4f0008000] <int> 
├─location = [312:0x4e8d40000] <chr> 
├─age_groups = [313:0x4e9a80000] <chr> 
├─ins_type = [314:0x4ea7c0000] <chr> 
└─age_decade = [315:0x4ec240000] <dbl>

set*() Functions

Memory-efficient operations by reference

The set*() family of functions modify data.table objects directly in memory without creating copies, making them essential for large datasets.

Why set functions are useful:

  1. Memory efficiency: No copies made, lower RAM usage
  2. Speed: Faster than traditional assignment operations
  3. Large data friendly: Essential when working with datasets that approach memory limits
  4. Consistent syntax: Uniform approach across different operations

Key set functions:

# setnames() - rename columns by reference
setnames(pums, "AGEP", "age") 
setnames(pums, c("old1", "old2"), c("new1", "new2"))

# setkey() - set keys for fast operations
setkey(pums, ST, year)  # Fast lookups and joins

# setorder() - sort by reference
setorder(pums, ST, -AGEP)  # Sort by state, then age descending

# set() - general purpose setter
set(pums, i = 1L, j = "AGEP", value = 99L)  # Set specific cell
set(pums, j = "new_col", value = 0L)        # Add column with default value

Tradeoffs of Modify by Reference

  1. Need to keep track of what is referencing the same data
  2. Use copies if you want to keep data independent
  3. When done properly, it can be very memory efficient

What will this do?

pums2 = pums
setnames(pums2, old = "SPORDER", new = "id")

What will this do?

pums2 = pums
setnames(pums2, old = "SPORDER", new = "id")
names(pums)
  [1] "id"         "RT"         "SERIALNO"   "PUMA"       "ST"        
  [6] "ADJUST"     "PWGTP"      "AGEP"       "CIT"        "COW"       
 [11] "DDRS"       "DEYE"       "DOUT"       "DPHY"       "DREM"      
 [16] "DWRK"       "ENG"        "FER"        "GCL"        "GCM"       
 [21] "GCR"        "INTP"       "JWMNP"      "JWRIP"      "JWTR"      
 [26] "LANX"       "MAR"        "MIG"        "MIL"        "MILY"      
 [31] "MLPA"       "MLPB"       "MLPC"       "MLPD"       "MLPE"      
 [36] "MLPF"       "MLPG"       "MLPH"       "MLPI"       "MLPJ"      
 [41] "MLPK"       "NWAB"       "NWAV"       "NWLA"       "NWLK"      
 [46] "NWRE"       "OIP"        "PAP"        "REL"        "RETP"      
 [51] "SCH"        "SCHG"       "SCHL"       "SEMP"       "SEX"       
 [56] "SSIP"       "SSP"        "WAGP"       "WKHP"       "WKL"       
 [61] "WKW"        "YOEP"       "UWRK"       "ANC"        "ANC1P"     
 [66] "ANC2P"      "DECADE"     "DRIVESP"    "DS"         "ESP"       
 [71] "ESR"        "HISP"       "INDP"       "JWAP"       "JWDP"      
 [76] "LANP"       "MIGPUMA"    "MIGSP"      "MSP"        "NAICSP"    
 [81] "NATIVITY"   "OC"         "OCCP"       "PAOC"       "PERNP"     
 [86] "PINCP"      "POBP"       "POVPIP"     "POWPUMA"    "POWSP"     
 [91] "QTRBIR"     "RAC1P"      "RAC2P"      "RAC3P"      "RACAIAN"   
 [96] "RACASN"     "RACBLK"     "RACNHPI"    "RACNUM"     "RACSOR"    
[101] "RACWHT"     "RC"         "SFN"        "SFR"        "SOCP"      
[106] "VPS"        "WAOB"       "FAGEP"      "FANCP"      "FCITP"     
[111] "FCOWP"      "FDDRSP"     "FDEYEP"     "FDOUTP"     "FDPHYP"    
[116] "FDREMP"     "FDWRKP"     "FENGP"      "FESRP"      "FFERP"     
[121] "FGCLP"      "FGCMP"      "FGCRP"      "FHISP"      "FINDP"     
[126] "FINTP"      "FJWDP"      "FJWMNP"     "FJWRIP"     "FJWTRP"    
[131] "FLANP"      "FLANXP"     "FMARP"      "FMIGP"      "FMIGSP"    
[136] "FMILPP"     "FMILSP"     "FMILYP"     "FOCCP"      "FOIP"      
[141] "FPAP"       "FPOBP"      "FPOWSP"     "FRACP"      "FRELP"     
[146] "FRETP"      "FSCHGP"     "FSCHLP"     "FSCHP"      "FSEMP"     
[151] "FSEXP"      "FSSIP"      "FSSP"       "FWAGP"      "FWKHP"     
[156] "FWKLP"      "FWKWP"      "FYOEP"      "PWGTP1"     "PWGTP2"    
[161] "PWGTP3"     "PWGTP4"     "PWGTP5"     "PWGTP6"     "PWGTP7"    
[166] "PWGTP8"     "PWGTP9"     "PWGTP10"    "PWGTP11"    "PWGTP12"   
[171] "PWGTP13"    "PWGTP14"    "PWGTP15"    "PWGTP16"    "PWGTP17"   
[176] "PWGTP18"    "PWGTP19"    "PWGTP20"    "PWGTP21"    "PWGTP22"   
[181] "PWGTP23"    "PWGTP24"    "PWGTP25"    "PWGTP26"    "PWGTP27"   
[186] "PWGTP28"    "PWGTP29"    "PWGTP30"    "PWGTP31"    "PWGTP32"   
[191] "PWGTP33"    "PWGTP34"    "PWGTP35"    "PWGTP36"    "PWGTP37"   
[196] "PWGTP38"    "PWGTP39"    "PWGTP40"    "PWGTP41"    "PWGTP42"   
[201] "PWGTP43"    "PWGTP44"    "PWGTP45"    "PWGTP46"    "PWGTP47"   
[206] "PWGTP48"    "PWGTP49"    "PWGTP50"    "PWGTP51"    "PWGTP52"   
[211] "PWGTP53"    "PWGTP54"    "PWGTP55"    "PWGTP56"    "PWGTP57"   
[216] "PWGTP58"    "PWGTP59"    "PWGTP60"    "PWGTP61"    "PWGTP62"   
[221] "PWGTP63"    "PWGTP64"    "PWGTP65"    "PWGTP66"    "PWGTP67"   
[226] "PWGTP68"    "PWGTP69"    "PWGTP70"    "PWGTP71"    "PWGTP72"   
[231] "PWGTP73"    "PWGTP74"    "PWGTP75"    "PWGTP76"    "PWGTP77"   
[236] "PWGTP78"    "PWGTP79"    "PWGTP80"    "NOP"        "ADJINC"    
[241] "CITWP"      "DEAR"       "DRAT"       "DRATX"      "HINS1"     
[246] "HINS2"      "HINS3"      "HINS4"      "HINS5"      "HINS6"     
[251] "HINS7"      "MARHD"      "MARHM"      "MARHT"      "MARHW"     
[256] "MARHYP"     "DIS"        "HICOV"      "PRIVCOV"    "PUBCOV"    
[261] "FCITWP"     "FDEARP"     "FDRATP"     "FDRATXP"    "FHINS1P"   
[266] "FHINS2P"    "FHINS3P"    "FHINS4P"    "FHINS5P"    "FHINS6P"   
[271] "FHINS7P"    "FMARHDP"    "FMARHMP"    "FMARHTP"    "FMARHWP"   
[276] "FMARHYP"    "WRK"        "FOD1P"      "FOD2P"      "SCIENGP"   
[281] "SCIENGRLP"  "FFODP"      "FHINS3C"    "FHINS4C"    "FHINS5C"   
[286] "RELP"       "FWRKP"      "FDISP"      "FPERNP"     "FPINCP"    
[291] "FPRIVCOVP"  "FPUBCOVP"   "RACNH"      "RACPI"      "SSPA"      
[296] "MLPCD"      "MLPFG"      "FHICOVP"    "DIVISION"   "REGION"    
[301] "HIMRKS"     "JWTRNS"     "RELSHIPP"   "WKWN"       "FHIMRKSP"  
[306] "FJWTRNSP"   "FRELSHIPP"  "FWKWNP"     "MLPIK"      "year"      
[311] "location"   "age_groups" "ins_type"   "age_decade"

What creates a copy besides copy()


  1. When you subset the data (select columns or filter rows)
  2. Change a data.table to a data.frame or tibble
  3. Aggregate or summarizing of the data.table

4. API Lifecycle Management

“I’m not superstitious, but I am a little stitious.”


Stable and backward compatible

Key principles:

  • Backward compatibility maintained
  • New features don’t break existing code
  • Clear deprecation warnings when needed

5. Community

Active development and strong ecosystem

  • New Governance: As part of a NSF grant, new expansive governance
  • GitHub: 3,500+ stars, active issues and PRs
  • CRAN: Over 2,000 reverse dependencies
  • Stack Overflow: 15,000+ questions tagged with data.table
  • Documentation: Comprehensive vignettes and examples
  • Extensions: dtplyr, data.table.express, tidyfast, tidytable, and more
# Getting help
?data.table
vignette("datatable-intro")
browseVignettes("data.table")

6. Feature Rich

Comprehensive data manipulation toolkit

Joins

# Load household data for comparison
household_data <- open_dataset(here::here("data/household")) |>
  filter(year >= 2020, location %in% c("ca", "tx", "ny")) |>
  collect()
setDT(household_data)

# Create state-year summary from household data
household_summary <- household_data[, 
  .(avg_income = mean(HINCP, na.rm = TRUE)), 
  by = .(ST, year)
]

# Join person and household summaries
pums[household_summary, on = c("ST", "year")] # merge(pums, household_summary, by = c("ST", "year"))
            id            RT      SERIALNO   PUMA          ST ADJUST PWGTP
         <int>        <fctr>        <char> <char>      <char>  <int> <int>
      1:     1 Person Record 2021GQ0000077  03301 New York/NY     NA    25
      2:     1 Person Record 2021GQ0000090  01600 New York/NY     NA    10
      3:     1 Person Record 2021GQ0000091  03212 New York/NY     NA    72
      4:     1 Person Record 2021GQ0000094  02100 New York/NY     NA    65
      5:     1 Person Record 2021GQ0000097  04102 New York/NY     NA     6
     ---                                                                  
1735551:     1 Person Record 2022HU0609672  04611    Texas/TX     NA   136
1735552:     2 Person Record 2022HU0609672  04611    Texas/TX     NA   140
1735553:     3 Person Record 2022HU0609672  04611    Texas/TX     NA   167
1735554:     1 Person Record 2022HU0609675  04302    Texas/TX     NA    20
1735555:     2 Person Record 2022HU0609675  04302    Texas/TX     NA    33
          AGEP                            CIT
         <int>                         <fctr>
      1:    56      Born in the United States
      2:    48      Born in the United States
      3:    17      Born in the United States
      4:    51      Born in the United States
      5:    85      Born in the United States
     ---                                     
1735551:    56 U.S. citizen by naturalization
1735552:    67 U.S. citizen by naturalization
1735553:    21      Born in the United States
1735554:    66      Born in the United States
1735555:    64      Born in the United States
                                                                                                       COW
                                                                                                    <fctr>
      1:                                                                                              <NA>
      2:                                                                                              <NA>
      3:                      Employee of a private not-for-profit, tax-exempt, or charitable organization
      4:                                                                                              <NA>
      5:                                                                                              <NA>
     ---                                                                                                  
1735551: Employee of a private for-profit co. or bus., or of an individual, for wages, salary, commissions
1735552: Employee of a private for-profit co. or bus., or of an individual, for wages, salary, commissions
1735553: Employee of a private for-profit co. or bus., or of an individual, for wages, salary, commissions
1735554:                                                                                              <NA>
1735555: Employee of a private for-profit co. or bus., or of an individual, for wages, salary, commissions
           DDRS   DEYE   DOUT   DPHY   DREM   DWRK       ENG    FER    GCL
         <lgcl> <lgcl> <lgcl> <lgcl> <lgcl> <lgcl>    <fctr> <lgcl> <lgcl>
      1:  FALSE  FALSE  FALSE  FALSE  FALSE     NA      <NA>     NA  FALSE
      2:   TRUE  FALSE   TRUE   TRUE   TRUE     NA      <NA>  FALSE  FALSE
      3:  FALSE  FALSE  FALSE  FALSE   TRUE     NA      <NA>     NA     NA
      4:  FALSE   TRUE   TRUE  FALSE  FALSE     NA      <NA>     NA  FALSE
      5:   TRUE  FALSE   TRUE   TRUE  FALSE     NA      <NA>     NA  FALSE
     ---                                                                  
1735551:  FALSE  FALSE  FALSE  FALSE   TRUE     NA      Well     NA  FALSE
1735552:  FALSE  FALSE  FALSE  FALSE  FALSE     NA      Well     NA  FALSE
1735553:  FALSE  FALSE  FALSE  FALSE  FALSE     NA Very well  FALSE     NA
1735554:   TRUE  FALSE   TRUE   TRUE   TRUE     NA      <NA>     NA  FALSE
1735555:  FALSE  FALSE  FALSE   TRUE   TRUE     NA      <NA>     NA  FALSE
            GCM    GCR  INTP JWMNP  JWRIP   JWTR   LANX
         <fctr> <lgcl> <int> <int> <char> <char> <lgcl>
      1:   <NA>     NA     0    NA   <NA>   <NA>  FALSE
      2:   <NA>     NA     0    NA   <NA>   <NA>  FALSE
      3:   <NA>     NA     0    NA   <NA>   <NA>  FALSE
      4:   <NA>     NA     0    NA   <NA>   <NA>  FALSE
      5:   <NA>     NA     0    NA   <NA>   <NA>  FALSE
     ---                                               
1735551:   <NA>     NA -2400    40      1   <NA>   TRUE
1735552:   <NA>     NA     0    30      1   <NA>   TRUE
1735553:   <NA>     NA     0    20      1   <NA>   TRUE
1735554:   <NA>     NA     0    NA   <NA>   <NA>  FALSE
1735555:   <NA>     NA     0    30      1   <NA>  FALSE
                                         MAR    MIG
                                      <fctr> <lgcl>
      1: Never married or under 15 years old   TRUE
      2: Never married or under 15 years old   TRUE
      3: Never married or under 15 years old  FALSE
      4: Never married or under 15 years old  FALSE
      5: Never married or under 15 years old   TRUE
     ---                                           
1735551:                             Married   TRUE
1735552:                             Married   TRUE
1735553: Never married or under 15 years old   TRUE
1735554:                             Married   TRUE
1735555:                             Married   TRUE
                                  MIL   MILY   MLPA   MLPB   MLPC   MLPD   MLPE
                               <fctr> <fctr> <lgcl> <lgcl> <lgcl> <lgcl> <lgcl>
      1: Never served in the military   <NA>     NA     NA     NA     NA     NA
      2: Never served in the military   <NA>     NA     NA     NA     NA     NA
      3: Never served in the military   <NA>     NA     NA     NA     NA     NA
      4: Never served in the military   <NA>     NA     NA     NA     NA     NA
      5: Never served in the military   <NA>     NA     NA     NA     NA     NA
     ---                                                                       
1735551: Never served in the military   <NA>     NA     NA     NA     NA     NA
1735552: Never served in the military   <NA>     NA     NA     NA     NA     NA
1735553: Never served in the military   <NA>     NA     NA     NA     NA     NA
1735554: Never served in the military   <NA>     NA     NA     NA     NA     NA
1735555: Never served in the military   <NA>     NA     NA     NA     NA     NA
           MLPF   MLPG   MLPH   MLPI   MLPJ   MLPK   NWAB              NWAV
         <lgcl> <lgcl> <lgcl> <lgcl> <lgcl> <lgcl> <lgcl>            <fctr>
      1:     NA     NA     NA     NA     NA     NA  FALSE    Did not report
      2:     NA     NA     NA     NA     NA     NA  FALSE    Did not report
      3:     NA     NA     NA     NA     NA     NA     NA    Did not report
      4:     NA     NA     NA     NA     NA     NA     NA    Did not report
      5:     NA     NA     NA     NA     NA     NA  FALSE    Did not report
     ---                                                                   
1735551:     NA     NA     NA     NA     NA     NA     NA    Did not report
1735552:     NA     NA     NA     NA     NA     NA     NA    Did not report
1735553:     NA     NA     NA     NA     NA     NA     NA    Did not report
1735554:     NA     NA     NA     NA     NA     NA  FALSE No, other reasons
1735555:     NA     NA     NA     NA     NA     NA  FALSE               Yes
           NWLA   NWLK   NWRE   OIP   PAP    REL  RETP    SCH
         <lgcl> <lgcl> <lgcl> <int> <int> <char> <int> <lgcl>
      1:  FALSE  FALSE     NA  2100     0   <NA>     0  FALSE
      2:  FALSE  FALSE     NA     0     0   <NA>     0  FALSE
      3:     NA     NA     NA     0     0   <NA>     0   TRUE
      4:     NA     NA     NA     0     0   <NA>     0  FALSE
      5:  FALSE  FALSE     NA 18200     0   <NA> 12000  FALSE
     ---                                                     
1735551:     NA     NA     NA     0     0   <NA>     0  FALSE
1735552:     NA     NA     NA     0     0   <NA>     0  FALSE
1735553:     NA     NA     NA     0     0   <NA>     0   TRUE
1735554:  FALSE  FALSE  FALSE     0     0   <NA>     0  FALSE
1735555:  FALSE  FALSE     NA     0     0   <NA>     0  FALSE
                                                     SCHG
                                                   <char>
      1:                                             <NA>
      2:                                             <NA>
      3: College undergraduate years (freshman to senior)
      4:                                             <NA>
      5:                                             <NA>
     ---                                                 
1735551:                                             <NA>
1735552:                                             <NA>
1735553: College undergraduate years (freshman to senior)
1735554:                                             <NA>
1735555:                                             <NA>
                                                 SCHL  SEMP    SEX  SSIP   SSP
                                               <char> <int> <fctr> <int> <int>
      1:                                     Grade 10     0   Male     0     0
      2:                GED or alternative credential     0 Female     0 13800
      3:           Some college, but less than 1 year     0   Male     0     0
      4:                                     Grade 10     0   Male     0     0
      5:                            Bachelor's degree     0   Male     0  6800
     ---                                                                      
1735551:                            Bachelor's degree     0 Female     0     0
1735552:                  Regular high school diploma     0   Male     0     0
1735553: 1 or more years of college credit, no degree     0 Female     0     0
1735554:           Some college, but less than 1 year     0   Male  3600 30100
1735555:                  Regular high school diploma 12100 Female     0     0
          WAGP  WKHP                              WKL    WKW   YOEP   UWRK
         <int> <int>                           <fctr> <fctr> <char> <lgcl>
      1:     0    NA Over 5 years ago or never worked   <NA>   <NA>     NA
      2:     0    NA Over 5 years ago or never worked   <NA>   <NA>     NA
      3:  5000    14        Within the past 12 months   <NA>   <NA>     NA
      4:     0    NA Over 5 years ago or never worked   <NA>   <NA>     NA
      5:     0    NA Over 5 years ago or never worked   <NA>   <NA>     NA
     ---                                                                  
1735551: 30000    40        Within the past 12 months   <NA>   1975     NA
1735552: 50000    40        Within the past 12 months   <NA>   1957     NA
1735553: 18000    20        Within the past 12 months   <NA>   <NA>     NA
1735554:     0    NA Over 5 years ago or never worked   <NA>   <NA>     NA
1735555: 30100    40        Within the past 12 months   <NA>   <NA>     NA
                  ANC            ANC1P        ANC2P      DECADE
               <fctr>           <char>       <char>      <fctr>
      1:       Single African American Not reported        <NA>
      2: Not reported     Not reported Not reported        <NA>
      3: Not reported     Not reported Not reported        <NA>
      4:       Single African American Not reported        <NA>
      5:       Single            Irish Not reported        <NA>
     ---                                                       
1735551:       Single          Chinese Not reported 1970 - 1979
1735552:       Single       Vietnamese Not reported 1950 - 1959
1735553:       Single       Vietnamese Not reported        <NA>
1735554:       Single            White Not reported        <NA>
1735555:       Single          Mexican Not reported        <NA>
                              DRIVESP     DS    ESP                        ESR
                               <fctr> <lgcl> <fctr>                     <fctr>
      1:                         <NA>     NA   <NA>         Not in Labor Force
      2:                         <NA>     NA   <NA>         Not in Labor Force
      3:                         <NA>     NA   <NA> Civilian employed, at work
      4:                         <NA>     NA   <NA>         Not in Labor Force
      5:                         <NA>     NA   <NA>         Not in Labor Force
     ---                                                                      
1735551: 1.000 vehicles (Drove alone)     NA   <NA> Civilian employed, at work
1735552: 1.000 vehicles (Drove alone)     NA   <NA> Civilian employed, at work
1735553: 1.000 vehicles (Drove alone)     NA   <NA> Civilian employed, at work
1735554:                         <NA>     NA   <NA>         Not in Labor Force
1735555: 1.000 vehicles (Drove alone)     NA   <NA> Civilian employed, at work
                                HISP
                              <char>
      1: Not Spanish/Hispanic/Latino
      2: Not Spanish/Hispanic/Latino
      3: Not Spanish/Hispanic/Latino
      4: Not Spanish/Hispanic/Latino
      5: Not Spanish/Hispanic/Latino
     ---                            
1735551: Not Spanish/Hispanic/Latino
1735552: Not Spanish/Hispanic/Latino
1735553: Not Spanish/Hispanic/Latino
1735554: Not Spanish/Hispanic/Latino
1735555:                     Mexican
                                                                                    INDP
                                                                                  <char>
      1:                                                                            <NA>
      2:                                                                            <NA>
      3: EDU-Colleges, Universities, And Professional Schools, Including Junior Colleges
      4:                                                                            <NA>
      5:                                                                            <NA>
     ---                                                                                
1735551:                                              FIN-Banking And Related Activities
1735552:                                                          RET-Automobile Dealers
1735553:      RET-General Merchandise Stores, Including Warehouse Clubs and Supercenters
1735554:                                                                            <NA>
1735555:                                            EDU-Elementary And Secondary Schools
                             JWAP                     JWDP       LANP MIGPUMA
                           <char>                   <char>     <char>  <fctr>
      1:                     <NA>                     <NA>       <NA>    <NA>
      2:                     <NA>                     <NA>       <NA>    <NA>
      3:                     <NA>                     <NA>       <NA>    <NA>
      4:                     <NA>                     <NA>       <NA>    <NA>
      5:                     <NA>                     <NA>       <NA>    <NA>
     ---                                                                     
1735551:   7:55 a.m. to 7:59 a.m.   7:15 a.m. to 7:19 a.m. Vietnamese    <NA>
1735552:   7:30 a.m. to 7:34 a.m.   7:00 a.m. to 7:04 a.m. Vietnamese    <NA>
1735553: 11:50 a.m. to 11:54 a.m. 11:30 a.m. to 11:39 a.m. Vietnamese    <NA>
1735554:                     <NA>                     <NA>       <NA>    <NA>
1735555:   7:00 a.m. to 7:04 a.m.   6:30 a.m. to 6:34 a.m.       <NA>    <NA>
               MIGSP                         MSP
              <char>                      <fctr>
      1:        <NA>               Never Married
      2:        <NA>               Never Married
      3: New York/NY               Never Married
      4: New York/NY               Never Married
      5:        <NA>               Never Married
     ---                                        
1735551:        <NA> Now Married, Spouse Present
1735552:        <NA> Now Married, Spouse Present
1735553:        <NA>               Never Married
1735554:        <NA> Now Married, Spouse Present
1735555:        <NA> Now Married, Spouse Present
                                                                                  NAICSP
                                                                                  <char>
      1:                                                                            <NA>
      2:                                                                            <NA>
      3: EDU-Colleges, Universities, And Professional Schools, Including Junior Colleges
      4:                                                                            <NA>
      5:                                                                            <NA>
     ---                                                                                
1735551:                                              FIN-Banking And Related Activities
1735552:                                                          RET-Automobile Dealers
1735553:      RET-General Merchandise Stores, Including Warehouse Clubs and Supercenters
1735554:                                                                            <NA>
1735555:                                            EDU-Elementary And Secondary Schools
             NATIVITY     OC
               <fctr> <lgcl>
      1:       Native     NA
      2:       Native     NA
      3:       Native     NA
      4:       Native     NA
      5:       Native     NA
     ---                    
1735551: Foreign born     NA
1735552: Foreign born     NA
1735553:       Native     NA
1735554:       Native     NA
1735555:       Native     NA
                                                          OCCP
                                                        <char>
      1:                                                  <NA>
      2:                                                  <NA>
      3: EDU-Other Educational Instruction and Library Workers
      4:                                                  <NA>
      5:                                                  <NA>
     ---                                                      
1735551:                                           OFF-Tellers
1735552:      RPR-Automotive Service Technicians And Mechanics
1735553:                              MED-Pharmacy Technicians
1735554:                                                  <NA>
1735555:                               EDU-Teaching Assistants
                                 PAOC PERNP PINCP         POBP POVPIP POWPUMA
                               <fctr> <int> <int>       <char>  <int>  <fctr>
      1:                         <NA>     0  2100  New York/NY     15    <NA>
      2:                         <NA>     0 13800  New York/NY    100    <NA>
      3:                         <NA>  5000  5000  Illinois/IL     NA    <NA>
      4:                         <NA>     0     0  New York/NY     NA    <NA>
      5:                         <NA>     0 37000  New York/NY    293    <NA>
     ---                                                                     
1735551: Females with no own children 30000 27600      Vietnam    419    <NA>
1735552:                         <NA> 50000 50000      Vietnam    419    <NA>
1735553: Females with no own children 18000 18000     Texas/TX    419    <NA>
1735554:                         <NA>     0 33700 Minnesota/MN    431    <NA>
1735555: Females with no own children 42200 42200     Texas/TX    431    <NA>
               POWSP                   QTRBIR                           RAC1P
              <char>                   <fctr>                          <fctr>
      1:        <NA> October through December Black or African American alone
      2:        <NA>    January through March                     White alone
      3: New York/NY       April through June                     White alone
      4:        <NA>    January through March Black or African American alone
      5:        <NA>       April through June                     White alone
     ---                                                                     
1735551:    Texas/TX October through December                     Asian alone
1735552:    Texas/TX       April through June                     Asian alone
1735553:    Texas/TX    January through March                     Asian alone
1735554:        <NA>       April through June                     White alone
1735555:    Texas/TX       April through June           Some other race alone
                                   RAC2P                           RAC3P
                                  <char>                          <char>
      1: Black or African American alone Black or African American alone
      2:                     White alone                     White alone
      3:                     White alone                     White alone
      4: Black or African American alone Black or African American alone
      5:                     White alone                     White alone
     ---                                                                
1735551:                Vietnamese alone                Vietnamese alone
1735552:                Vietnamese alone                Vietnamese alone
1735553:                Vietnamese alone                Vietnamese alone
1735554:                     White alone                     White alone
1735555:           Some Other Race alone           Some Other Race alone
         RACAIAN RACASN RACBLK RACNHPI RACNUM RACSOR RACWHT     RC    SFN
          <lgcl> <lgcl> <lgcl>  <lgcl>  <int> <lgcl> <lgcl> <lgcl> <fctr>
      1:   FALSE  FALSE   TRUE      NA      1  FALSE  FALSE     NA   <NA>
      2:   FALSE  FALSE  FALSE      NA      1  FALSE   TRUE     NA   <NA>
      3:   FALSE  FALSE  FALSE      NA      1  FALSE   TRUE     NA   <NA>
      4:   FALSE  FALSE   TRUE      NA      1  FALSE  FALSE     NA   <NA>
      5:   FALSE  FALSE  FALSE      NA      1  FALSE   TRUE     NA   <NA>
     ---                                                                 
1735551:   FALSE   TRUE  FALSE      NA      1  FALSE  FALSE     NA   <NA>
1735552:   FALSE   TRUE  FALSE      NA      1  FALSE  FALSE     NA   <NA>
1735553:   FALSE   TRUE  FALSE      NA      1  FALSE  FALSE     NA   <NA>
1735554:   FALSE  FALSE  FALSE      NA      1  FALSE   TRUE     NA   <NA>
1735555:   FALSE  FALSE  FALSE      NA      1   TRUE  FALSE     NA   <NA>
            SFR                                                  SOCP    VPS
         <fctr>                                                <char> <char>
      1:   <NA>                                                  <NA>   <NA>
      2:   <NA>                                                  <NA>   <NA>
      3:   <NA> EDU-Other Educational Instruction and Library Workers   <NA>
      4:   <NA>                                                  <NA>   <NA>
      5:   <NA>                                                  <NA>   <NA>
     ---                                                                    
1735551:   <NA>                                           OFF-Tellers   <NA>
1735552:   <NA>      RPR-Automotive Service Technicians And Mechanics   <NA>
1735553:   <NA>                              MED-Pharmacy Technicians   <NA>
1735554:   <NA>                                                  <NA>   <NA>
1735555:   <NA>                               EDU-Teaching Assistants   <NA>
                                      WAOB  FAGEP  FANCP  FCITP  FCOWP FDDRSP
                                    <fctr> <lgcl> <lgcl> <lgcl> <lgcl> <lgcl>
      1:         US state (POBP = 001-059)  FALSE  FALSE  FALSE  FALSE  FALSE
      2:         US state (POBP = 001-059)  FALSE  FALSE  FALSE  FALSE  FALSE
      3:         US state (POBP = 001-059)  FALSE  FALSE   TRUE   TRUE   TRUE
      4:         US state (POBP = 001-059)  FALSE  FALSE  FALSE  FALSE  FALSE
      5:         US state (POBP = 001-059)  FALSE  FALSE  FALSE  FALSE  FALSE
     ---                                                                     
1735551: Asia (POBP = 158-159,161,200-299)  FALSE  FALSE  FALSE  FALSE  FALSE
1735552: Asia (POBP = 158-159,161,200-299)  FALSE  FALSE  FALSE  FALSE  FALSE
1735553:         US state (POBP = 001-059)  FALSE  FALSE  FALSE  FALSE  FALSE
1735554:         US state (POBP = 001-059)  FALSE  FALSE  FALSE  FALSE  FALSE
1735555:         US state (POBP = 001-059)  FALSE  FALSE  FALSE  FALSE  FALSE
         FDEYEP FDOUTP FDPHYP FDREMP FDWRKP  FENGP  FESRP  FFERP  FGCLP  FGCMP
         <lgcl> <lgcl> <lgcl> <lgcl> <lgcl> <lgcl> <lgcl> <lgcl> <lgcl> <lgcl>
      1:  FALSE  FALSE  FALSE  FALSE     NA  FALSE  FALSE  FALSE  FALSE  FALSE
      2:  FALSE  FALSE  FALSE  FALSE     NA  FALSE  FALSE  FALSE  FALSE  FALSE
      3:   TRUE   TRUE   TRUE   TRUE     NA  FALSE   TRUE  FALSE  FALSE  FALSE
      4:  FALSE  FALSE  FALSE   TRUE     NA  FALSE  FALSE  FALSE  FALSE  FALSE
      5:  FALSE  FALSE  FALSE  FALSE     NA  FALSE  FALSE  FALSE  FALSE  FALSE
     ---                                                                      
1735551:  FALSE  FALSE  FALSE  FALSE     NA  FALSE  FALSE  FALSE  FALSE  FALSE
1735552:  FALSE  FALSE  FALSE  FALSE     NA  FALSE  FALSE  FALSE  FALSE  FALSE
1735553:  FALSE  FALSE  FALSE  FALSE     NA  FALSE  FALSE  FALSE  FALSE  FALSE
1735554:  FALSE  FALSE  FALSE  FALSE     NA  FALSE  FALSE  FALSE  FALSE  FALSE
1735555:  FALSE  FALSE  FALSE  FALSE     NA  FALSE  FALSE  FALSE  FALSE  FALSE
          FGCRP  FHISP  FINDP  FINTP  FJWDP FJWMNP FJWRIP FJWTRP  FLANP FLANXP
         <lgcl> <lgcl> <lgcl> <lgcl> <lgcl> <lgcl> <lgcl> <lgcl> <lgcl> <lgcl>
      1:  FALSE  FALSE  FALSE  FALSE  FALSE  FALSE  FALSE     NA  FALSE  FALSE
      2:  FALSE  FALSE  FALSE  FALSE  FALSE  FALSE  FALSE     NA  FALSE  FALSE
      3:  FALSE  FALSE   TRUE   TRUE  FALSE  FALSE  FALSE     NA  FALSE   TRUE
      4:  FALSE  FALSE  FALSE   TRUE  FALSE  FALSE  FALSE     NA  FALSE  FALSE
      5:  FALSE  FALSE  FALSE   TRUE  FALSE  FALSE  FALSE     NA  FALSE  FALSE
     ---                                                                      
1735551:  FALSE  FALSE  FALSE  FALSE  FALSE  FALSE  FALSE     NA  FALSE  FALSE
1735552:  FALSE  FALSE  FALSE  FALSE  FALSE  FALSE  FALSE     NA  FALSE  FALSE
1735553:  FALSE  FALSE  FALSE  FALSE  FALSE  FALSE  FALSE     NA  FALSE  FALSE
1735554:  FALSE  FALSE  FALSE  FALSE  FALSE  FALSE  FALSE     NA  FALSE  FALSE
1735555:  FALSE  FALSE  FALSE  FALSE  FALSE  FALSE  FALSE     NA  FALSE   TRUE
          FMARP  FMIGP FMIGSP FMILPP FMILSP FMILYP  FOCCP   FOIP   FPAP  FPOBP
         <lgcl> <lgcl> <lgcl> <lgcl> <lgcl> <lgcl> <lgcl> <lgcl> <lgcl> <lgcl>
      1:  FALSE  FALSE  FALSE  FALSE  FALSE     NA  FALSE  FALSE  FALSE  FALSE
      2:  FALSE  FALSE  FALSE  FALSE  FALSE     NA  FALSE  FALSE  FALSE  FALSE
      3:   TRUE   TRUE   TRUE  FALSE   TRUE     NA   TRUE   TRUE   TRUE   TRUE
      4:  FALSE  FALSE   TRUE  FALSE   TRUE     NA  FALSE   TRUE   TRUE  FALSE
      5:  FALSE  FALSE  FALSE  FALSE  FALSE     NA  FALSE   TRUE   TRUE  FALSE
     ---                                                                      
1735551:  FALSE  FALSE  FALSE  FALSE  FALSE     NA  FALSE  FALSE  FALSE  FALSE
1735552:  FALSE  FALSE  FALSE  FALSE  FALSE     NA  FALSE  FALSE  FALSE  FALSE
1735553:  FALSE  FALSE  FALSE  FALSE  FALSE     NA  FALSE  FALSE  FALSE  FALSE
1735554:  FALSE  FALSE  FALSE  FALSE  FALSE     NA  FALSE  FALSE  FALSE  FALSE
1735555:  FALSE  FALSE  FALSE  FALSE  FALSE     NA  FALSE  FALSE  FALSE  FALSE
         FPOWSP  FRACP  FRELP  FRETP FSCHGP FSCHLP  FSCHP  FSEMP  FSEXP  FSSIP
         <lgcl> <lgcl> <lgcl> <lgcl> <lgcl> <lgcl> <lgcl> <lgcl> <lgcl> <lgcl>
      1:  FALSE  FALSE     NA  FALSE  FALSE  FALSE  FALSE  FALSE  FALSE  FALSE
      2:  FALSE  FALSE     NA  FALSE  FALSE  FALSE  FALSE  FALSE  FALSE  FALSE
      3:  FALSE  FALSE     NA   TRUE   TRUE   TRUE  FALSE   TRUE  FALSE   TRUE
      4:  FALSE  FALSE     NA   TRUE  FALSE  FALSE  FALSE   TRUE  FALSE   TRUE
      5:  FALSE  FALSE     NA  FALSE  FALSE  FALSE  FALSE  FALSE  FALSE   TRUE
     ---                                                                      
1735551:  FALSE  FALSE     NA  FALSE  FALSE  FALSE  FALSE  FALSE  FALSE  FALSE
1735552:  FALSE  FALSE     NA  FALSE  FALSE  FALSE  FALSE  FALSE  FALSE  FALSE
1735553:  FALSE  FALSE     NA  FALSE  FALSE  FALSE  FALSE  FALSE  FALSE  FALSE
1735554:  FALSE  FALSE     NA  FALSE  FALSE  FALSE  FALSE  FALSE  FALSE   TRUE
1735555:  FALSE  FALSE     NA  FALSE  FALSE  FALSE  FALSE  FALSE  FALSE  FALSE
           FSSP  FWAGP  FWKHP  FWKLP  FWKWP  FYOEP PWGTP1 PWGTP2 PWGTP3 PWGTP4
         <lgcl> <lgcl> <lgcl> <lgcl> <lgcl> <lgcl>  <int>  <int>  <int>  <int>
      1:  FALSE  FALSE  FALSE  FALSE     NA  FALSE     23     23      1     24
      2:   TRUE  FALSE  FALSE  FALSE     NA  FALSE     10     18      1      9
      3:   TRUE   TRUE   TRUE   TRUE     NA  FALSE     10    122     77    133
      4:   TRUE   TRUE  FALSE   TRUE     NA  FALSE     67    117    121    117
      5:   TRUE  FALSE  FALSE  FALSE     NA  FALSE      7      0     11      6
     ---                                                                      
1735551:  FALSE  FALSE  FALSE  FALSE     NA   TRUE     35    231    139    217
1735552:  FALSE  FALSE  FALSE  FALSE     NA   TRUE     41    236    143    221
1735553:  FALSE  FALSE  FALSE  FALSE     NA  FALSE     50    285    180    237
1735554:  FALSE  FALSE  FALSE  FALSE     NA  FALSE      5     31      6     16
1735555:  FALSE  FALSE  FALSE  FALSE     NA  FALSE      6     65      9     27
         PWGTP5 PWGTP6 PWGTP7 PWGTP8 PWGTP9 PWGTP10 PWGTP11 PWGTP12 PWGTP13
          <int>  <int>  <int>  <int>  <int>   <int>   <int>   <int>   <int>
      1:     45      2      1     49     23      22      48      24      46
      2:      2     10     20      9     18       0      10      17      18
      3:     72      9     78     10    138      76      10      10      75
      4:     61     68     65     68     62      15      14      15      63
      5:      1      1      7      5      6       5       4      10       5
     ---                                                                   
1735551:    149     36    153     44    198     123      40      42     139
1735552:    157     35    139     41    196     132      44      46     151
1735553:    179     46    206     54    250     133      50      53     167
1735554:     25     28     31     41     23      16       5      40      17
1735555:     45     42     64     52     35      28       7      49      31
         PWGTP14 PWGTP15 PWGTP16 PWGTP17 PWGTP18 PWGTP19 PWGTP20 PWGTP21
           <int>   <int>   <int>   <int>   <int>   <int>   <int>   <int>
      1:      26      24      25       2      23       2      44      22
      2:       8      10      12       9      10       2       2       9
      3:      68      80      65      73     128      69     134       9
      4:     118      69      15      68      65      15      17      61
      5:       1      11       5      11       6       0      11       7
     ---                                                                
1735551:     160     130     142     117     170     162      53     187
1735552:     152     141     161     117     183     168      53     216
1735553:     178     152     187     142     212     232      54     267
1735554:       5      23      22      24      17       6       6      26
1735555:      10      30      33      40      25      10      12      38
         PWGTP22 PWGTP23 PWGTP24 PWGTP25 PWGTP26 PWGTP27 PWGTP28 PWGTP29
           <int>   <int>   <int>   <int>   <int>   <int>   <int>   <int>
      1:      23       3      25      46       2       2      48      25
      2:      18       2      10       2       9      17      10      18
      3:     126      77     139      68      10      70      10     136
      4:      15      14      15      67      60      64      65      66
      5:       0      10       5       0       1       5       5       5
     ---                                                                
1735551:      49     134      40     179     218     141     198      36
1735552:      53     141      42     169     206     139     208      36
1735553:      70     164      38     202     218     167     256      40
1735554:      19      25      29       6      18      18      16      34
1735555:      26      47      46      10      46      27      28      63
         PWGTP30 PWGTP31 PWGTP32 PWGTP33 PWGTP34 PWGTP35 PWGTP36 PWGTP37
           <int>   <int>   <int>   <int>   <int>   <int>   <int>   <int>
      1:      23      44      26      44      24      24      23       3
      2:       2       9      20      17      10       9       9      10
      3:      73      12      11      71      67      72      74      75
      4:     118     111     111      66      15      66     116      65
      5:       6       5      11       5       0      11       5      11
     ---                                                                
1735551:     127     224     190     132     141     141     161     138
1735552:     129     231     203     137     126     145     172     129
1735553:     172     278     243     197     177     177     204     153
1735554:       6      16      21      34      19      40      34       7
1735555:      13      25      41      89      33      58      59      12
         PWGTP38 PWGTP39 PWGTP40 PWGTP41 PWGTP42 PWGTP43 PWGTP44 PWGTP45
           <int>   <int>   <int>   <int>   <int>   <int>   <int>   <int>
      1:      23       3       1      23      23      45      24       1
      2:       9       2      18       9       1      18      10      18
      3:     130      70     141      11     126      71     123      79
      4:      70     112      15      65      15      15      15      64
      5:       6       0      10       6       1      11       5       2
     ---                                                                
1735551:      33     163     275      41     221     141     210     131
1735552:      35     156     288      39     248     140     209     142
1735553:      41     188     336      46     263     164     255     156
1735554:       5      22      20       6      40       8      16      18
1735555:       7      38      34       8      54       8      27      42
         PWGTP46 PWGTP47 PWGTP48 PWGTP49 PWGTP50 PWGTP51 PWGTP52 PWGTP53
           <int>   <int>   <int>   <int>   <int>   <int>   <int>   <int>
      1:      48      48       1      24      24       1      22       1
      2:      10       1       9       0      19       8       0       1
      3:      13      67      12     136      66      11      11      73
      4:      66      65      68      61     114     124     113      64
      5:       0       5       6       5       7       5      10       6
     ---                                                                
1735551:      38     154      47     230     121      41      34     107
1735552:      40     159      45     223     140      44      35     105
1735553:      48     181      59     240     151      47      41     143
1735554:      33      31      29      22      17       5      37      17
1735555:      61      54      43      33      19       9      77      31
         PWGTP54 PWGTP55 PWGTP56 PWGTP57 PWGTP58 PWGTP59 PWGTP60 PWGTP61
           <int>   <int>   <int>   <int>   <int>   <int>   <int>   <int>
      1:      25      24      24      45      25      46       1      24
      2:      10      10       9      10       9      17      19      10
      3:      76      70      77      76     129      70     139      10
      4:      14      68     123      62      64     114     117      66
      5:       1      10       4      10       6       0      11       7
     ---                                                                
1735551:     145     156     142     145     209     141      41     201
1735552:     144     156     151     154     196     152      50     209
1735553:     179     175     196     158     249     211      58     300
1735554:       5      20      21      18      18       8       7      23
1735555:      10      50      28      29      31      13      11      50
         PWGTP62 PWGTP63 PWGTP64 PWGTP65 PWGTP66 PWGTP67 PWGTP68 PWGTP69
           <int>   <int>   <int>   <int>   <int>   <int>   <int>   <int>
      1:      24      47      24       3      47      46       1      24
      2:       1      17       9      18      10       2       9       1
      3:     122      68     130      71      13      66      10     130
      4:     118     114     124      70      66      64      65      69
      5:       1      11       5       1       0       5       5       6
     ---                                                                
1735551:      46     148      39     127     193     121     255      38
1735552:      49     158      37     134     192     130     268      41
1735553:      64     167      40     181     233     162     327      49
1735554:      15      19      29       5      21      26      20      32
1735555:      22      36      37       9      40      39      25      47
         PWGTP70 PWGTP71 PWGTP72 PWGTP73 PWGTP74 PWGTP75 PWGTP76 PWGTP77
           <int>   <int>   <int>   <int>   <int>   <int>   <int>   <int>
      1:      23       3      25       1      24      24      23      43
      2:      17       9       1       0      10      10       9       9
      3:      77      12      12      67      79      76      71      71
      4:      15      13      14      64     119      67      13      63
      5:       6       6      11       5       0      11       5      10
     ---                                                                
1735551:     124     208     208     124     139     172     156     151
1735552:     135     193     212     123     128     159     169     186
1735553:     150     277     220     174     172     183     227     208
1735554:       4      16      21      40      20      36      42       7
1735555:       8      30      52      67      32      82      67      16
         PWGTP78 PWGTP79 PWGTP80    NOP   ADJINC  CITWP   DEAR   DRAT  DRATX
           <int>   <int>   <int> <fctr>    <num> <char> <lgcl> <fctr> <lgcl>
      1:      24      48      46   <NA> 1.029928   <NA>  FALSE   <NA>     NA
      2:      10      19       0   <NA> 1.029928   <NA>  FALSE   <NA>     NA
      3:     138      71     120   <NA> 1.029928   <NA>  FALSE   <NA>     NA
      4:      68      15     110   <NA> 1.029928   <NA>   TRUE   <NA>     NA
      5:       6       0      10   <NA> 1.029928   <NA>   TRUE   <NA>     NA
     ---                                                                    
1735551:      43     130     245   <NA> 1.042311   1989  FALSE   <NA>     NA
1735552:      42     138     275   <NA> 1.042311   1993  FALSE   <NA>     NA
1735553:      50     158     344   <NA> 1.042311   <NA>  FALSE   <NA>     NA
1735554:       5      19      22   <NA> 1.042311   <NA>   TRUE   <NA>     NA
1735555:       9      46      44   <NA> 1.042311   <NA>  FALSE   <NA>     NA
          HINS1  HINS2  HINS3  HINS4  HINS5  HINS6  HINS7  MARHD  MARHM
         <lgcl> <lgcl> <lgcl> <lgcl> <lgcl> <lgcl> <lgcl> <lgcl> <lgcl>
      1:  FALSE  FALSE  FALSE   TRUE  FALSE  FALSE  FALSE     NA     NA
      2:  FALSE  FALSE   TRUE   TRUE  FALSE  FALSE  FALSE     NA     NA
      3:   TRUE  FALSE  FALSE  FALSE  FALSE  FALSE  FALSE     NA     NA
      4:  FALSE  FALSE  FALSE   TRUE  FALSE  FALSE  FALSE     NA     NA
      5:   TRUE  FALSE   TRUE  FALSE  FALSE  FALSE  FALSE     NA     NA
     ---                                                               
1735551:   TRUE  FALSE  FALSE  FALSE  FALSE  FALSE  FALSE  FALSE  FALSE
1735552:   TRUE   TRUE  FALSE  FALSE  FALSE  FALSE  FALSE  FALSE  FALSE
1735553:  FALSE   TRUE  FALSE  FALSE  FALSE  FALSE  FALSE     NA     NA
1735554:   TRUE  FALSE   TRUE   TRUE  FALSE  FALSE  FALSE  FALSE  FALSE
1735555:   TRUE  FALSE  FALSE  FALSE  FALSE  FALSE  FALSE  FALSE  FALSE
            MARHT  MARHW MARHYP    DIS  HICOV PRIVCOV PUBCOV FCITWP FDEARP
           <fctr> <lgcl> <char> <lgcl> <lgcl>  <lgcl> <lgcl> <lgcl> <lgcl>
      1:     <NA>     NA   <NA>  FALSE   TRUE   FALSE   TRUE  FALSE  FALSE
      2:     <NA>     NA   <NA>   TRUE   TRUE   FALSE   TRUE  FALSE  FALSE
      3:     <NA>     NA   <NA>   TRUE   TRUE    TRUE  FALSE  FALSE   TRUE
      4:     <NA>     NA   <NA>   TRUE   TRUE   FALSE   TRUE  FALSE  FALSE
      5:     <NA>     NA   <NA>   TRUE   TRUE    TRUE   TRUE  FALSE  FALSE
     ---                                                                  
1735551: One time  FALSE   1998   TRUE   TRUE    TRUE  FALSE  FALSE  FALSE
1735552: One time  FALSE   1998  FALSE   TRUE    TRUE  FALSE  FALSE  FALSE
1735553:     <NA>     NA   <NA>  FALSE   TRUE    TRUE  FALSE  FALSE  FALSE
1735554: One time  FALSE   1974   TRUE   TRUE    TRUE   TRUE  FALSE  FALSE
1735555: One time  FALSE   1974   TRUE   TRUE    TRUE  FALSE  FALSE  FALSE
         FDRATP FDRATXP FHINS1P FHINS2P FHINS3P FHINS4P FHINS5P FHINS6P FHINS7P
         <lgcl>  <lgcl>  <lgcl>  <lgcl>  <lgcl>  <lgcl>  <lgcl>  <lgcl>  <lgcl>
      1:  FALSE   FALSE   FALSE   FALSE   FALSE   FALSE   FALSE   FALSE   FALSE
      2:  FALSE   FALSE   FALSE   FALSE   FALSE   FALSE   FALSE   FALSE   FALSE
      3:  FALSE   FALSE    TRUE    TRUE    TRUE    TRUE    TRUE    TRUE    TRUE
      4:  FALSE   FALSE   FALSE   FALSE   FALSE   FALSE   FALSE   FALSE   FALSE
      5:  FALSE   FALSE   FALSE   FALSE   FALSE   FALSE   FALSE   FALSE   FALSE
     ---                                                                       
1735551:  FALSE   FALSE   FALSE   FALSE   FALSE   FALSE   FALSE   FALSE   FALSE
1735552:  FALSE   FALSE   FALSE   FALSE   FALSE   FALSE   FALSE   FALSE   FALSE
1735553:  FALSE   FALSE   FALSE   FALSE   FALSE   FALSE   FALSE   FALSE   FALSE
1735554:  FALSE   FALSE   FALSE    TRUE   FALSE   FALSE    TRUE    TRUE    TRUE
1735555:  FALSE   FALSE   FALSE   FALSE   FALSE   FALSE   FALSE   FALSE   FALSE
         FMARHDP FMARHMP FMARHTP FMARHWP FMARHYP    WRK            FOD1P  FOD2P
          <lgcl>  <lgcl>  <lgcl>  <lgcl>  <lgcl> <lgcl>           <char> <char>
      1:   FALSE   FALSE   FALSE   FALSE   FALSE  FALSE             <NA>   <NA>
      2:   FALSE   FALSE   FALSE   FALSE   FALSE  FALSE             <NA>   <NA>
      3:   FALSE   FALSE   FALSE   FALSE   FALSE     NA             <NA>   <NA>
      4:   FALSE   FALSE   FALSE   FALSE   FALSE     NA             <NA>   <NA>
      5:   FALSE   FALSE   FALSE   FALSE   FALSE  FALSE        Sociology   <NA>
     ---                                                                       
1735551:   FALSE   FALSE   FALSE   FALSE   FALSE   TRUE General Business   <NA>
1735552:   FALSE   FALSE   FALSE   FALSE   FALSE   TRUE             <NA>   <NA>
1735553:   FALSE   FALSE   FALSE   FALSE   FALSE   TRUE             <NA>   <NA>
1735554:   FALSE   FALSE   FALSE   FALSE   FALSE  FALSE             <NA>   <NA>
1735555:   FALSE   FALSE   FALSE   FALSE   FALSE   TRUE             <NA>   <NA>
         SCIENGP SCIENGRLP  FFODP FHINS3C FHINS4C FHINS5C   RELP  FWRKP  FDISP
          <lgcl>    <lgcl> <lgcl>  <fctr>  <fctr>  <fctr> <char> <lgcl> <lgcl>
      1:      NA        NA  FALSE    <NA>      No    <NA>   <NA>  FALSE  FALSE
      2:      NA        NA  FALSE      No      No    <NA>   <NA>  FALSE  FALSE
      3:      NA        NA  FALSE    <NA>    <NA>    <NA>   <NA>  FALSE   TRUE
      4:      NA        NA  FALSE    <NA>      No    <NA>   <NA>  FALSE   TRUE
      5:    TRUE     FALSE  FALSE      No    <NA>    <NA>   <NA>  FALSE  FALSE
     ---                                                                      
1735551:   FALSE     FALSE  FALSE    <NA>    <NA>    <NA>   <NA>  FALSE  FALSE
1735552:      NA        NA  FALSE    <NA>    <NA>    <NA>   <NA>  FALSE  FALSE
1735553:      NA        NA  FALSE    <NA>    <NA>    <NA>   <NA>  FALSE  FALSE
1735554:      NA        NA  FALSE      No     Yes    <NA>   <NA>  FALSE  FALSE
1735555:      NA        NA  FALSE    <NA>    <NA>    <NA>   <NA>  FALSE  FALSE
         FPERNP FPINCP FPRIVCOVP FPUBCOVP  RACNH  RACPI   SSPA  MLPCD  MLPFG
         <lgcl> <lgcl>    <lgcl>   <lgcl> <lgcl> <lgcl> <fctr> <lgcl> <lgcl>
      1:  FALSE  FALSE     FALSE    FALSE  FALSE  FALSE   <NA>     NA     NA
      2:  FALSE   TRUE     FALSE    FALSE  FALSE  FALSE   <NA>     NA     NA
      3:   TRUE   TRUE      TRUE     TRUE  FALSE  FALSE   <NA>     NA     NA
      4:   TRUE   TRUE     FALSE    FALSE  FALSE  FALSE   <NA>     NA     NA
      5:  FALSE   TRUE     FALSE    FALSE  FALSE  FALSE   <NA>     NA     NA
     ---                                                                    
1735551:  FALSE  FALSE     FALSE    FALSE  FALSE  FALSE   <NA>     NA     NA
1735552:  FALSE  FALSE     FALSE    FALSE  FALSE  FALSE   <NA>     NA     NA
1735553:  FALSE  FALSE     FALSE    FALSE  FALSE  FALSE   <NA>     NA     NA
1735554:  FALSE   TRUE      TRUE     TRUE  FALSE  FALSE   <NA>     NA     NA
1735555:  FALSE  FALSE     FALSE    FALSE  FALSE  FALSE   <NA>     NA     NA
         FHICOVP                           DIVISION    REGION
          <lgcl>                             <fctr>    <fctr>
      1:   FALSE Middle Atlantic (Northeast region) Northeast
      2:   FALSE Middle Atlantic (Northeast region) Northeast
      3:    TRUE Middle Atlantic (Northeast region) Northeast
      4:   FALSE Middle Atlantic (Northeast region) Northeast
      5:   FALSE Middle Atlantic (Northeast region) Northeast
     ---                                                     
1735551:   FALSE  West South Central (South Region)     South
1735552:   FALSE  West South Central (South Region)     South
1735553:   FALSE  West South Central (South Region)     South
1735554:    TRUE  West South Central (South Region)     South
1735555:   FALSE  West South Central (South Region)     South
                                                      HIMRKS             JWTRNS
                                                      <fctr>             <char>
      1:                Categorically ineligible for subsidy               <NA>
      2:                Categorically ineligible for subsidy               <NA>
      3:                Categorically ineligible for subsidy   Worked from home
      4:                Categorically ineligible for subsidy               <NA>
      5:                Categorically ineligible for subsidy               <NA>
     ---                                                                       
1735551:                Categorically ineligible for subsidy Car, truck, or van
1735552: Direct purchase coverage, without a premium subsidy Car, truck, or van
1735553: Direct purchase coverage, without a premium subsidy Car, truck, or van
1735554:                Categorically ineligible for subsidy               <NA>
1735555:                Categorically ineligible for subsidy Car, truck, or van
                                               RELSHIPP  WKWN FHIMRKSP FJWTRNSP
                                                 <char> <int>   <lgcl>   <lgcl>
      1: Noninstitutionalized group quarters population    NA    FALSE    FALSE
      2: Noninstitutionalized group quarters population    NA    FALSE    FALSE
      3: Noninstitutionalized group quarters population    40    FALSE     TRUE
      4:    Institutionalized group quarters population    NA    FALSE    FALSE
      5: Noninstitutionalized group quarters population    NA    FALSE    FALSE
     ---                                                                       
1735551:                               Reference person    52    FALSE    FALSE
1735552:               Opposite-sex husband/wife/spouse    52    FALSE    FALSE
1735553:                     Biological son or daughter    52    FALSE    FALSE
1735554:                               Reference person    NA    FALSE    FALSE
1735555:               Opposite-sex husband/wife/spouse    37    FALSE    FALSE
         FRELSHIPP FWKWNP  MLPIK  year location age_groups ins_type age_decade
            <lgcl> <lgcl> <lgcl> <int>   <char>     <char>   <char>      <num>
      1:     FALSE  FALSE     NA  2021       ny      18-64 Medicaid         50
      2:     FALSE  FALSE     NA  2021       ny      18-64 Medicare         40
      3:     FALSE   TRUE     NA  2021       ny   Under 18 employer         10
      4:     FALSE  FALSE     NA  2021       ny      18-64 Medicaid         50
      5:     FALSE  FALSE     NA  2021       ny        65+ employer         80
     ---                                                                      
1735551:     FALSE  FALSE     NA  2022       tx      18-64 employer         50
1735552:     FALSE  FALSE     NA  2022       tx        65+ employer         60
1735553:     FALSE  FALSE     NA  2022       tx      18-64   direct         20
1735554:     FALSE  FALSE     NA  2022       tx        65+ employer         60
1735555:     FALSE  FALSE     NA  2022       tx      18-64 employer         60
         avg_income
              <num>
      1:   109935.7
      2:   109935.7
      3:   109935.7
      4:   109935.7
      5:   109935.7
     ---           
1735551:   103775.2
1735552:   103775.2
1735553:   103775.2
1735554:   103775.2
1735555:   103775.2

Feature Rich

Reshaping

# Pivot longer - select key demographics
pums_long <- melt(
  pums, 
  id.vars = c("ST", "year", "SERIALNO"),
  measure.vars = c("AGEP", "PWGTP", "PINCP")
)
pums_long
                  ST  year      SERIALNO variable value
              <char> <int>        <char>   <fctr> <int>
      1: New York/NY  2021 2021GQ0000077     AGEP    56
      2: New York/NY  2021 2021GQ0000090     AGEP    48
      3: New York/NY  2021 2021GQ0000091     AGEP    17
      4: New York/NY  2021 2021GQ0000094     AGEP    51
      5: New York/NY  2021 2021GQ0000097     AGEP    85
     ---                                               
5206661:    Texas/TX  2022 2022HU0609672    PINCP 27600
5206662:    Texas/TX  2022 2022HU0609672    PINCP 50000
5206663:    Texas/TX  2022 2022HU0609672    PINCP 18000
5206664:    Texas/TX  2022 2022HU0609675    PINCP 33700
5206665:    Texas/TX  2022 2022HU0609675    PINCP 42200
# Pivot wider - average by state/year
pums_summary <- pums[, 
  .(avg_age = mean(AGEP, na.rm = TRUE),
    avg_income = mean(PINCP, na.rm = TRUE)), 
  by = .(ST, year)
]
dcast(pums_summary, ST ~ year, value.var = "avg_age")
Key: <ST>
              ST     2021     2022
          <char>    <num>    <num>
1: California/CA 41.47709 41.54009
2:   New York/NY 43.12366 43.18311
3:      Texas/TX 40.59856 40.82080

Feature Rich

Rolling operations

# Population change over time by state
pums_yearly <- pums[, .(pop = sum(PWGTP)), by = .(ST, year)]
setkey(pums_yearly, ST, year)

# Calculate 2-year rolling average (limited years available)
pums_yearly[, pop_2yr_avg := frollmean(pop, 2), by = ST]
pums_yearly
Key: <ST, year>
              ST  year      pop pop_2yr_avg
          <char> <int>    <int>       <num>
1: California/CA  2021 39237836          NA
2: California/CA  2022 39029342    39133589
3:   New York/NY  2021 19835913          NA
4:   New York/NY  2022 19677151    19756532
5:      Texas/TX  2021 29527941          NA
6:      Texas/TX  2022 30029572    29778756

Feature Rich

Rolling Joins

benefit_thresholds <- data.table(
  age_threshold = c(18, 25, 35, 50, 62, 67),
  benefit_type = c("Youth", "Young Adult", "Adult", "Mid-Career", "Pre-Retirement", "Senior"),
  max_benefit = c(500, 800, 1200, 1500, 2000, 2500)
)
setkey(benefit_thresholds, age_threshold)
benefit_thresholds
Key: <age_threshold>
   age_threshold   benefit_type max_benefit
           <num>         <char>       <num>
1:            18          Youth         500
2:            25    Young Adult         800
3:            35          Adult        1200
4:            50     Mid-Career        1500
5:            62 Pre-Retirement        2000
6:            67         Senior        2500

. . .

# Rolling join to assign benefits based on nearest age threshold
pums[
  benefit_thresholds, 
  benefit_category := benefit_type, 
  on = .(AGEP = age_threshold), 
  roll = -Inf
]
pums[, .(AGEP, benefit_category, ins_type, location)]
          AGEP benefit_category ins_type location
         <int>           <char>   <char>   <char>
      1:    56             <NA> Medicaid       ny
      2:    48             <NA> Medicare       ny
      3:    17             <NA> employer       ny
      4:    51             <NA> Medicaid       ny
      5:    85             <NA> employer       ny
     ---                                         
1735551:    56             <NA> employer       tx
1735552:    67           Senior employer       tx
1735553:    21             <NA>   direct       tx
1735554:    66             <NA> employer       tx
1735555:    64             <NA> employer       tx

roll = "nearest" or roll = +Inf are also possible

Feature Rich

Operators - Core Symbols

Essential data.table operators for efficient data manipulation

# .N - Number of rows in each group
pums[, .N, by = ST]  # Count rows by state
              ST      N
          <char>  <int>
1:   New York/NY 403958
2:      Texas/TX 554365
3: California/CA 777232
# .SD - Subset of Data (all columns except grouping)
pums[, names(.SD) := lapply(.SD, mean, na.rm = TRUE), by = ST, .SDcols = c("AGEP", "PINCP")]
pums[, .(ST, AGEP, PINCP)]
                  ST  AGEP PINCP
              <char> <int> <int>
      1: New York/NY    43 52025
      2: New York/NY    43 52025
      3: New York/NY    43 52025
      4: New York/NY    43 52025
      5: New York/NY    43 52025
     ---                        
1735551:    Texas/TX    40 45756
1735552:    Texas/TX    40 45756
1735553:    Texas/TX    40 45756
1735554:    Texas/TX    40 45756
1735555:    Texas/TX    40 45756
# .BY - List of grouping variables
pums[, .(mean_age = mean(AGEP), state = gsub("/.*$", "", .BY)), by = ST]
              ST mean_age      state
          <char>    <num>     <char>
1:   New York/NY       43   New York
2:      Texas/TX       40      Texas
3: California/CA       41 California
# .I - Row indices 
pums[, .I[which.max(AGEP)], by = ST]  # Index of oldest person per state
              ST     V1
          <char>  <int>
1:   New York/NY      1
2:      Texas/TX 200068
3: California/CA 232836

Feature Rich

Operators - Advanced Features

Specialized operators for complex operations

# .EACHI - For each item in i (useful with joins)
household_summary[pums, on = .(ST, year), mean(AGEP), by = .EACHI]
                  ST  year    V1
              <char> <int> <num>
      1: New York/NY  2021    43
      2: New York/NY  2021    43
      3: New York/NY  2021    43
      4: New York/NY  2021    43
      5: New York/NY  2021    43
     ---                        
1735551:    Texas/TX  2022    40
1735552:    Texas/TX  2022    40
1735553:    Texas/TX  2022    40
1735554:    Texas/TX  2022    40
1735555:    Texas/TX  2022    40
# .GRP - Group counter (1, 2, 3, ...)
pums[, .(grp_num = .GRP, pop = sum(PWGTP)), by = ST]
              ST grp_num      pop
          <char>   <int>    <int>
1:   New York/NY       1 39513064
2:      Texas/TX       2 59557513
3: California/CA       3 78267178
# Multiple .SD operations with .SDcols
pums[, {
    means <- lapply(.SD, mean, na.rm = TRUE)
    medians <- lapply(.SD, median, na.rm = TRUE)
    vars <- names(.SD)
    .(vars, means, medians)
  }, 
  by = ST, 
  .SDcols = c("AGEP", "PINCP", "PWGTP")
]
              ST   vars    means medians
          <char> <char>   <list>  <list>
1:   New York/NY   AGEP       43      43
2:   New York/NY  PINCP    52025   52025
3:   New York/NY  PWGTP 97.81478      73
4:      Texas/TX   AGEP       40      40
5:      Texas/TX  PINCP    45756   45756
6:      Texas/TX  PWGTP 107.4338      76
7: California/CA   AGEP       41      41
8: California/CA  PINCP    54155   54155
9: California/CA  PWGTP 100.6999      75
# Chaining with operators
pums[year == 2022][, .N, by = ST][order(-N)]
              ST      N
          <char>  <int>
1: California/CA 391171
2:      Texas/TX 292919
3:   New York/NY 203891

Feature Rich

Not convinced by the syntax?

library(dtplyr)

pums_dt <- pums |> 
  lazy_dt() |> 
  select(PINCP, age_groups, year) |> 
  group_by(age_groups, year) |> 
  summarise(avg_tot_income = mean(PINCP)) |> 
  collect()  # or as.data.table() will collect it too
pums_dt
# A tibble: 6 × 3
# Groups:   age_groups [3]
  age_groups  year avg_tot_income
  <chr>      <int>          <dbl>
1 18-64       2021         51100.
2 18-64       2022         50939.
3 65+         2021         51131.
4 65+         2022         50953.
5 Under 18    2021         50873.
6 Under 18    2022         50701.
pums |> 
  lazy_dt() |> 
  select(PINCP, age_groups, year) |> 
  group_by(age_groups, year) |> 
  summarise(avg_tot_income = mean(PINCP)) |> 
  show_query()
`_DT2`[, .(PINCP, age_groups, year)][, .(avg_tot_income = mean(PINCP)), 
    keyby = .(age_groups, year)]

Review

  1. Learn the DT[i, j, by] syntax - it’s the foundation
  2. Use keys for fast subsetting and joins
  3. Modify by reference with := for memory efficiency
  4. Chain operations and use operators for readable, efficient code
  5. Check the documentation - excellent vignettes available

Resources:

  • Official vignettes: vignette("datatable-intro")
  • GitHub: https://github.com/Rdatatable/data.table
  • Stack Overflow: [data.table] tag